INFO: Started server process [1] INFO: Waiting for application startup. INFO: Application startup complete. INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit) INFO: 172.21.0.1:44474 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54734 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55048 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40682 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57372 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42068 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47382 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38572 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34676 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41550 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60794 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55982 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49178 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54328 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59144 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59168 - "GET / HTTP/1.0" 200 OK 2026-03-24 00:29:08,656 - lava_callback - INFO - Received callback for node 69c1da520efaf21d93103585 2026-03-24 00:29:08,668 - lava_callback - INFO - Starting async job submit for node 69c1da520efaf21d93103585 INFO: 172.21.0.1:59176 - "POST /node/69c1da520efaf21d93103585 HTTP/1.0" 202 Accepted 2026-03-24 00:29:08,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:29:08,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1da520efaf21d93103585 HTTP/1.1" 200 None 2026-03-24 00:29:08,713 - lava_callback - DEBUG - Processing node 69c1da520efaf21d93103585: baseline-arm INFO: 172.21.0.1:59188 - "GET / HTTP/1.0" 200 OK 2026-03-24 00:29:08,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:29:09,137 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:29:09,139 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c1da520efaf21d93103585/log.txt.gz 2026-03-24 00:29:09,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:29:09,475 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:29:09,476 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c1da520efaf21d93103585/lava-logs.yaml 2026-03-24 00:29:09,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:29:09,652 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:29:09,653 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c1da520efaf21d93103585/lava_callback.json.gz 2026-03-24 00:29:09,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:29:09,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1da520efaf21d93103585 HTTP/1.1" 200 None 2026-03-24 00:29:09,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:29:09,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8340efaf21d93103549 HTTP/1.1" 200 None 2026-03-24 00:29:09,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:29:09,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1da520efaf21d93103585 HTTP/1.1" 200 None 2026-03-24 00:29:09,764 - lava_callback - INFO - Completed processing callback for node 69c1da520efaf21d93103585 INFO: 172.21.0.1:58308 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52534 - "GET / HTTP/1.0" 200 OK 2026-03-24 00:29:39,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:29:39,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:41618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35618 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48960 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53482 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37866 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34920 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52518 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51734 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37340 - "GET / HTTP/1.0" 200 OK 2026-03-24 00:36:42,674 - lava_callback - INFO - Received callback for node 69c1dc900efaf21d931035a8 INFO: 172.21.0.1:49742 - "POST /node/69c1dc900efaf21d931035a8 HTTP/1.0" 202 Accepted 2026-03-24 00:36:42,680 - lava_callback - INFO - Starting async job submit for node 69c1dc900efaf21d931035a8 2026-03-24 00:36:42,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:36:42,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dc900efaf21d931035a8 HTTP/1.1" 200 None 2026-03-24 00:36:42,698 - lava_callback - DEBUG - Processing node 69c1dc900efaf21d931035a8: baseline-arm 2026-03-24 00:36:42,703 - lava_callback - WARNING - Failed to upload log 2026-03-24 00:36:42,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:36:42,893 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:36:42,895 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c1dc900efaf21d931035a8/lava-logs.yaml 2026-03-24 00:36:42,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:36:43,016 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:36:43,017 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c1dc900efaf21d931035a8/lava_callback.json.gz 2026-03-24 00:36:43,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:36:43,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dc900efaf21d931035a8 HTTP/1.1" 200 None 2026-03-24 00:36:43,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:36:43,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-24 00:36:43,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:36:43,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1dc900efaf21d931035a8 HTTP/1.1" 200 None 2026-03-24 00:36:43,083 - lava_callback - INFO - Completed processing callback for node 69c1dc900efaf21d931035a8 INFO: 172.21.0.1:49748 - "GET / HTTP/1.0" 200 OK 2026-03-24 00:36:54,344 - lava_callback - INFO - Received callback for node 69c1dc9b0efaf21d931035d2 INFO: 172.21.0.1:35060 - "POST /node/69c1dc9b0efaf21d931035d2 HTTP/1.0" 202 Accepted 2026-03-24 00:36:54,354 - lava_callback - INFO - Starting async job submit for node 69c1dc9b0efaf21d931035d2 2026-03-24 00:36:54,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:36:54,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dc9b0efaf21d931035d2 HTTP/1.1" 200 None 2026-03-24 00:36:54,378 - lava_callback - DEBUG - Processing node 69c1dc9b0efaf21d931035d2: baseline-arm 2026-03-24 00:36:54,383 - lava_callback - WARNING - Failed to upload log 2026-03-24 00:36:54,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:36:54,553 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:36:54,555 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c1dc9b0efaf21d931035d2/lava-logs.yaml 2026-03-24 00:36:54,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:36:54,742 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:36:54,744 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c1dc9b0efaf21d931035d2/lava_callback.json.gz 2026-03-24 00:36:54,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:36:54,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dc9b0efaf21d931035d2 HTTP/1.1" 200 None 2026-03-24 00:36:54,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:36:54,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-24 00:36:54,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:36:54,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1dc9b0efaf21d931035d2 HTTP/1.1" 200 None 2026-03-24 00:36:54,839 - lava_callback - INFO - Completed processing callback for node 69c1dc9b0efaf21d931035d2 INFO: 172.21.0.1:35074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41512 - "GET / HTTP/1.0" 200 OK 2026-03-24 00:37:07,147 - lava_callback - INFO - Received callback for node 69c1dc940efaf21d931035b4 INFO: 172.21.0.1:41520 - "POST /node/69c1dc940efaf21d931035b4 HTTP/1.0" 202 Accepted 2026-03-24 00:37:07,194 - lava_callback - INFO - Starting async job submit for node 69c1dc940efaf21d931035b4 2026-03-24 00:37:07,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:37:07,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dc940efaf21d931035b4 HTTP/1.1" 200 None 2026-03-24 00:37:07,227 - lava_callback - DEBUG - Processing node 69c1dc940efaf21d931035b4: baseline-arm 2026-03-24 00:37:07,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:37:07,581 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:37:07,583 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c1dc940efaf21d931035b4/log.txt.gz 2026-03-24 00:37:07,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:37:07,827 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:37:07,829 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c1dc940efaf21d931035b4/lava-logs.yaml 2026-03-24 00:37:07,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:37:07,986 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:37:07,988 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c1dc940efaf21d931035b4/lava_callback.json.gz 2026-03-24 00:37:08,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:37:08,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dc940efaf21d931035b4 HTTP/1.1" 200 None 2026-03-24 00:37:08,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:37:08,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-24 00:37:08,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:37:08,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1dc940efaf21d931035b4 HTTP/1.1" 200 None 2026-03-24 00:37:08,146 - lava_callback - INFO - Completed processing callback for node 69c1dc940efaf21d931035b4 INFO: 172.21.0.1:41524 - "GET / HTTP/1.0" 200 OK 2026-03-24 00:37:09,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:37:09,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-24 00:37:11,550 - lava_callback - INFO - Received callback for node 69c1dc910efaf21d931035ab INFO: 172.21.0.1:36326 - "POST /node/69c1dc910efaf21d931035ab HTTP/1.0" 202 Accepted 2026-03-24 00:37:11,598 - lava_callback - INFO - Starting async job submit for node 69c1dc910efaf21d931035ab 2026-03-24 00:37:11,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:37:11,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dc910efaf21d931035ab HTTP/1.1" 200 None 2026-03-24 00:37:11,632 - lava_callback - DEBUG - Processing node 69c1dc910efaf21d931035ab: baseline-arm 2026-03-24 00:37:11,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:37:11,924 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:37:11,926 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c1dc910efaf21d931035ab/log.txt.gz 2026-03-24 00:37:12,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:37:12,151 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:37:12,153 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c1dc910efaf21d931035ab/lava-logs.yaml 2026-03-24 00:37:12,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:37:12,313 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:37:12,316 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c1dc910efaf21d931035ab/lava_callback.json.gz 2026-03-24 00:37:12,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:37:12,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dc910efaf21d931035ab HTTP/1.1" 200 None 2026-03-24 00:37:12,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:37:12,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-24 00:37:12,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:37:12,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1dc910efaf21d931035ab HTTP/1.1" 200 None 2026-03-24 00:37:12,441 - lava_callback - INFO - Completed processing callback for node 69c1dc910efaf21d931035ab INFO: 172.21.0.1:36336 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52488 - "GET / HTTP/1.0" 200 OK 2026-03-24 00:37:39,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:37:39,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:53178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35486 - "GET / HTTP/1.0" 200 OK 2026-03-24 00:38:01,756 - lava_callback - INFO - Received callback for node 69c1dc930efaf21d931035b1 INFO: 172.21.0.1:36104 - "POST /node/69c1dc930efaf21d931035b1 HTTP/1.0" 202 Accepted 2026-03-24 00:38:01,762 - lava_callback - INFO - Starting async job submit for node 69c1dc930efaf21d931035b1 2026-03-24 00:38:01,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:38:01,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dc930efaf21d931035b1 HTTP/1.1" 200 None 2026-03-24 00:38:01,798 - lava_callback - DEBUG - Processing node 69c1dc930efaf21d931035b1: baseline-arm 2026-03-24 00:38:02,030 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:38:02,217 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:38:02,219 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c1dc930efaf21d931035b1/log.txt.gz 2026-03-24 00:38:02,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:38:02,549 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:38:02,550 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c1dc930efaf21d931035b1/lava-logs.yaml 2026-03-24 00:38:02,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:38:02,683 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:38:02,685 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c1dc930efaf21d931035b1/lava_callback.json.gz 2026-03-24 00:38:02,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:38:02,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dc930efaf21d931035b1 HTTP/1.1" 200 None 2026-03-24 00:38:02,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:38:02,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-24 00:38:02,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:38:02,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1dc930efaf21d931035b1 HTTP/1.1" 200 None 2026-03-24 00:38:02,808 - lava_callback - INFO - Completed processing callback for node 69c1dc930efaf21d931035b1 INFO: 172.21.0.1:36118 - "GET / HTTP/1.0" 200 OK 2026-03-24 00:38:06,796 - lava_callback - INFO - Received callback for node 69c1dc900efaf21d931035a5 INFO: 172.21.0.1:36124 - "POST /node/69c1dc900efaf21d931035a5 HTTP/1.0" 202 Accepted 2026-03-24 00:38:06,802 - lava_callback - INFO - Starting async job submit for node 69c1dc900efaf21d931035a5 2026-03-24 00:38:06,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:38:06,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dc900efaf21d931035a5 HTTP/1.1" 200 None 2026-03-24 00:38:06,839 - lava_callback - DEBUG - Processing node 69c1dc900efaf21d931035a5: baseline-arm 2026-03-24 00:38:07,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:38:07,149 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:38:07,151 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c1dc900efaf21d931035a5/log.txt.gz 2026-03-24 00:38:07,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:38:07,448 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:38:07,450 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c1dc900efaf21d931035a5/lava-logs.yaml 2026-03-24 00:38:07,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:38:07,582 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:38:07,584 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c1dc900efaf21d931035a5/lava_callback.json.gz 2026-03-24 00:38:07,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:38:07,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dc900efaf21d931035a5 HTTP/1.1" 200 None 2026-03-24 00:38:07,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:38:07,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-24 00:38:07,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:38:07,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1dc900efaf21d931035a5 HTTP/1.1" 200 None 2026-03-24 00:38:07,675 - lava_callback - INFO - Completed processing callback for node 69c1dc900efaf21d931035a5 INFO: 172.21.0.1:36132 - "GET / HTTP/1.0" 200 OK 2026-03-24 00:38:09,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:38:09,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:41514 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-24 00:38:15,168 - lava_callback - INFO - Received callback for node 69c1dcb80efaf21d93103631 INFO: 172.21.0.1:41520 - "POST /node/69c1dcb80efaf21d93103631 HTTP/1.0" 202 Accepted 2026-03-24 00:38:15,217 - lava_callback - INFO - Starting async job submit for node 69c1dcb80efaf21d93103631 2026-03-24 00:38:15,239 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:38:15,254 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dcb80efaf21d93103631 HTTP/1.1" 200 None 2026-03-24 00:38:15,255 - lava_callback - DEBUG - Processing node 69c1dcb80efaf21d93103631: baseline-arm-pengutronix 2026-03-24 00:38:15,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:38:15,485 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:38:15,487 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c1dcb80efaf21d93103631/log.txt.gz 2026-03-24 00:38:15,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:38:15,692 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:38:15,694 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c1dcb80efaf21d93103631/lava-logs.yaml 2026-03-24 00:38:15,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:38:15,870 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:38:15,872 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c1dcb80efaf21d93103631/lava_callback.json.gz 2026-03-24 00:38:15,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:38:15,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dcb80efaf21d93103631 HTTP/1.1" 200 None 2026-03-24 00:38:15,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:38:15,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-24 00:38:15,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:38:15,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1dcb80efaf21d93103631 HTTP/1.1" 200 None 2026-03-24 00:38:15,968 - lava_callback - INFO - Completed processing callback for node 69c1dcb80efaf21d93103631 INFO: 172.21.0.1:41530 - "GET / HTTP/1.0" 200 OK 2026-03-24 00:38:20,614 - lava_callback - INFO - Received callback for node 69c1dca50efaf21d931035ec INFO: 172.21.0.1:56862 - "POST /node/69c1dca50efaf21d931035ec HTTP/1.0" 202 Accepted 2026-03-24 00:38:20,620 - lava_callback - INFO - Starting async job submit for node 69c1dca50efaf21d931035ec 2026-03-24 00:38:20,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:38:20,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dca50efaf21d931035ec HTTP/1.1" 200 None 2026-03-24 00:38:20,659 - lava_callback - DEBUG - Processing node 69c1dca50efaf21d931035ec: baseline-arm-broonie 2026-03-24 00:38:20,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:38:21,003 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:38:21,005 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c1dca50efaf21d931035ec/log.txt.gz 2026-03-24 00:38:21,171 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:38:21,328 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:38:21,330 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c1dca50efaf21d931035ec/lava-logs.yaml 2026-03-24 00:38:21,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:38:21,470 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:38:21,472 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c1dca50efaf21d931035ec/lava_callback.json.gz 2026-03-24 00:38:21,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:38:21,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dca50efaf21d931035ec HTTP/1.1" 200 None 2026-03-24 00:38:21,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:38:21,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-24 00:38:21,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:38:21,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1dca50efaf21d931035ec HTTP/1.1" 200 None 2026-03-24 00:38:21,557 - lava_callback - INFO - Completed processing callback for node 69c1dca50efaf21d931035ec INFO: 172.21.0.1:56864 - "GET / HTTP/1.0" 200 OK 2026-03-24 00:38:28,864 - lava_callback - INFO - Received callback for node 69c1dca90efaf21d931035fd INFO: 172.21.0.1:56880 - "POST /node/69c1dca90efaf21d931035fd HTTP/1.0" 202 Accepted 2026-03-24 00:38:28,871 - lava_callback - INFO - Starting async job submit for node 69c1dca90efaf21d931035fd 2026-03-24 00:38:28,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:38:28,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dca90efaf21d931035fd HTTP/1.1" 200 None 2026-03-24 00:38:28,909 - lava_callback - DEBUG - Processing node 69c1dca90efaf21d931035fd: baseline-arm-clabbe 2026-03-24 00:38:29,089 - lava_callback - INFO - Received callback for node 69c1dca20efaf21d931035e6 2026-03-24 00:38:29,110 - lava_callback - INFO - Starting async job submit for node 69c1dca20efaf21d931035e6 INFO: 172.21.0.1:56896 - "POST /node/69c1dca20efaf21d931035e6 HTTP/1.0" 202 Accepted 2026-03-24 00:38:29,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:38:29,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dca20efaf21d931035e6 HTTP/1.1" 200 None 2026-03-24 00:38:29,282 - lava_callback - DEBUG - Processing node 69c1dca20efaf21d931035e6: baseline-arm-broonie 2026-03-24 00:38:29,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:38:29,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:38:30,018 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:38:30,020 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c1dca90efaf21d931035fd/log.txt.gz 2026-03-24 00:38:30,036 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:38:30,045 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c1dca20efaf21d931035e6/log.txt.gz 2026-03-24 00:38:30,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:38:30,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:38:30,851 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:38:30,853 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c1dca90efaf21d931035fd/lava-logs.yaml 2026-03-24 00:38:30,855 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:38:30,857 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c1dca20efaf21d931035e6/lava-logs.yaml 2026-03-24 00:38:30,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:38:30,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:38:31,000 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:38:31,002 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c1dca20efaf21d931035e6/lava_callback.json.gz 2026-03-24 00:38:31,029 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:38:31,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:38:31,042 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c1dca90efaf21d931035fd/lava_callback.json.gz 2026-03-24 00:38:31,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:38:31,098 - lava_callback - INFO - Received callback for node 69c1dca30efaf21d931035e9 2026-03-24 00:38:31,105 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dca20efaf21d931035e6 HTTP/1.1" 200 None 2026-03-24 00:38:31,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:38:31,118 - lava_callback - INFO - Starting async job submit for node 69c1dca30efaf21d931035e9 INFO: 172.21.0.1:37986 - "POST /node/69c1dca30efaf21d931035e9 HTTP/1.0" 202 Accepted 2026-03-24 00:38:31,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dca90efaf21d931035fd HTTP/1.1" 200 None 2026-03-24 00:38:31,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:38:31,166 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-24 00:38:31,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:38:31,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:38:31,220 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-24 00:38:31,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:38:31,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1dca20efaf21d931035e6 HTTP/1.1" 200 None 2026-03-24 00:38:31,245 - lava_callback - INFO - Completed processing callback for node 69c1dca20efaf21d931035e6 2026-03-24 00:38:31,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dca30efaf21d931035e9 HTTP/1.1" 200 None 2026-03-24 00:38:31,261 - lava_callback - DEBUG - Processing node 69c1dca30efaf21d931035e9: baseline-arm-broonie 2026-03-24 00:38:31,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1dca90efaf21d931035fd HTTP/1.1" 200 None 2026-03-24 00:38:31,389 - lava_callback - INFO - Completed processing callback for node 69c1dca90efaf21d931035fd 2026-03-24 00:38:31,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:38:31,979 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:38:31,981 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c1dca30efaf21d931035e9/log.txt.gz 2026-03-24 00:38:32,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:38:32,571 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:38:32,573 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c1dca30efaf21d931035e9/lava-logs.yaml 2026-03-24 00:38:32,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:38:32,716 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:38:32,718 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c1dca30efaf21d931035e9/lava_callback.json.gz 2026-03-24 00:38:32,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:38:32,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dca30efaf21d931035e9 HTTP/1.1" 200 None 2026-03-24 00:38:32,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:38:32,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-24 00:38:32,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:38:32,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1dca30efaf21d931035e9 HTTP/1.1" 200 None 2026-03-24 00:38:32,807 - lava_callback - INFO - Completed processing callback for node 69c1dca30efaf21d931035e9 INFO: 172.21.0.1:37990 - "GET / HTTP/1.0" 200 OK 2026-03-24 00:38:39,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:38:39,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.21.0.1:44892 - "GET / HTTP/1.0" 200 OK 2026-03-24 00:38:54,260 - lava_callback - INFO - Received callback for node 69c1dd0f0efaf21d931036e7 INFO: 172.21.0.1:48844 - "POST /node/69c1dd0f0efaf21d931036e7 HTTP/1.0" 202 Accepted 2026-03-24 00:38:54,266 - lava_callback - INFO - Starting async job submit for node 69c1dd0f0efaf21d931036e7 2026-03-24 00:38:54,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:38:54,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dd0f0efaf21d931036e7 HTTP/1.1" 200 None 2026-03-24 00:38:54,284 - lava_callback - DEBUG - Processing node 69c1dd0f0efaf21d931036e7: baseline-arm 2026-03-24 00:38:54,289 - lava_callback - WARNING - Failed to upload log 2026-03-24 00:38:54,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:38:54,430 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:38:54,432 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c1dd0f0efaf21d931036e7/lava-logs.yaml 2026-03-24 00:38:54,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:38:54,548 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:38:54,550 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c1dd0f0efaf21d931036e7/lava_callback.json.gz 2026-03-24 00:38:54,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:38:54,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dd0f0efaf21d931036e7 HTTP/1.1" 200 None 2026-03-24 00:38:54,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:38:54,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-24 00:38:54,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:38:54,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1dd0f0efaf21d931036e7 HTTP/1.1" 200 None 2026-03-24 00:38:54,608 - lava_callback - INFO - Completed processing callback for node 69c1dd0f0efaf21d931036e7 INFO: 172.21.0.1:48858 - "GET / HTTP/1.0" 200 OK 2026-03-24 00:39:00,374 - lava_callback - INFO - Received callback for node 69c1dca00efaf21d931035e3 INFO: 172.21.0.1:49014 - "POST /node/69c1dca00efaf21d931035e3 HTTP/1.0" 202 Accepted 2026-03-24 00:39:00,380 - lava_callback - INFO - Starting async job submit for node 69c1dca00efaf21d931035e3 2026-03-24 00:39:00,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:39:00,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dca00efaf21d931035e3 HTTP/1.1" 200 None 2026-03-24 00:39:00,417 - lava_callback - DEBUG - Processing node 69c1dca00efaf21d931035e3: baseline-arm-broonie 2026-03-24 00:39:00,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:39:00,764 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:39:00,766 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c1dca00efaf21d931035e3/log.txt.gz 2026-03-24 00:39:00,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:39:01,052 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:39:01,054 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c1dca00efaf21d931035e3/lava-logs.yaml 2026-03-24 00:39:01,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:39:01,228 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:39:01,230 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c1dca00efaf21d931035e3/lava_callback.json.gz 2026-03-24 00:39:01,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:39:01,254 - lava_callback - INFO - Received callback for node 69c1dcb50efaf21d93103625 INFO: 172.21.0.1:49020 - "POST /node/69c1dcb50efaf21d93103625 HTTP/1.0" 202 Accepted 2026-03-24 00:39:01,260 - lava_callback - INFO - Starting async job submit for node 69c1dcb50efaf21d93103625 2026-03-24 00:39:01,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:39:01,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dca00efaf21d931035e3 HTTP/1.1" 200 None 2026-03-24 00:39:01,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:39:01,302 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dcb50efaf21d93103625 HTTP/1.1" 200 None 2026-03-24 00:39:01,303 - lava_callback - DEBUG - Processing node 69c1dcb50efaf21d93103625: baseline-arm-pengutronix 2026-03-24 00:39:01,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-24 00:39:01,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:39:01,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:39:01,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1dca00efaf21d931035e3 HTTP/1.1" 200 None 2026-03-24 00:39:01,492 - lava_callback - INFO - Completed processing callback for node 69c1dca00efaf21d931035e3 2026-03-24 00:39:01,719 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:39:01,721 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c1dcb50efaf21d93103625/log.txt.gz 2026-03-24 00:39:01,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:39:01,996 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:39:01,998 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c1dcb50efaf21d93103625/lava-logs.yaml 2026-03-24 00:39:02,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:39:02,128 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:39:02,130 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c1dcb50efaf21d93103625/lava_callback.json.gz 2026-03-24 00:39:02,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:39:02,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dcb50efaf21d93103625 HTTP/1.1" 200 None 2026-03-24 00:39:02,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:39:02,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-24 00:39:02,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:39:02,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1dcb50efaf21d93103625 HTTP/1.1" 200 None 2026-03-24 00:39:02,274 - lava_callback - INFO - Completed processing callback for node 69c1dcb50efaf21d93103625 2026-03-24 00:39:02,431 - lava_callback - INFO - Received callback for node 69c1dcb40efaf21d93103622 INFO: 172.21.0.1:49036 - "POST /node/69c1dcb40efaf21d93103622 HTTP/1.0" 202 Accepted 2026-03-24 00:39:02,440 - lava_callback - INFO - Starting async job submit for node 69c1dcb40efaf21d93103622 2026-03-24 00:39:02,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:39:02,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dcb40efaf21d93103622 HTTP/1.1" 200 None 2026-03-24 00:39:02,493 - lava_callback - DEBUG - Processing node 69c1dcb40efaf21d93103622: baseline-arm-pengutronix 2026-03-24 00:39:02,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:39:02,779 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:39:02,781 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c1dcb40efaf21d93103622/log.txt.gz 2026-03-24 00:39:02,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:39:03,078 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:39:03,080 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c1dcb40efaf21d93103622/lava-logs.yaml 2026-03-24 00:39:03,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:39:03,215 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:39:03,217 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c1dcb40efaf21d93103622/lava_callback.json.gz 2026-03-24 00:39:03,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:39:03,292 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dcb40efaf21d93103622 HTTP/1.1" 200 None 2026-03-24 00:39:03,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:39:03,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-24 00:39:03,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:39:03,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1dcb40efaf21d93103622 HTTP/1.1" 200 None 2026-03-24 00:39:03,397 - lava_callback - INFO - Completed processing callback for node 69c1dcb40efaf21d93103622 INFO: 172.21.0.1:49046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49062 - "GET / HTTP/1.0" 200 OK 2026-03-24 00:39:09,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:39:09,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.21.0.1:48772 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44216 - "GET / HTTP/1.0" 200 OK 2026-03-24 00:39:32,278 - lava_callback - INFO - Received callback for node 69c1dccf0efaf21d93103672 INFO: 172.21.0.1:40896 - "POST /node/69c1dccf0efaf21d93103672 HTTP/1.0" 202 Accepted 2026-03-24 00:39:32,285 - lava_callback - INFO - Starting async job submit for node 69c1dccf0efaf21d93103672 2026-03-24 00:39:32,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:39:32,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dccf0efaf21d93103672 HTTP/1.1" 200 None 2026-03-24 00:39:32,325 - lava_callback - DEBUG - Processing node 69c1dccf0efaf21d93103672: ltp-dio 2026-03-24 00:39:32,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:39:32,838 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:39:32,840 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-dio-69c1dccf0efaf21d93103672/log.txt.gz 2026-03-24 00:39:33,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:39:33,291 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:39:33,293 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-dio-69c1dccf0efaf21d93103672/lava-logs.yaml 2026-03-24 00:39:33,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:39:33,488 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:39:33,490 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-dio-69c1dccf0efaf21d93103672/lava_callback.json.gz 2026-03-24 00:39:33,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:39:33,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dccf0efaf21d93103672 HTTP/1.1" 200 None 2026-03-24 00:39:33,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:39:33,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-24 00:39:33,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:39:33,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1dccf0efaf21d93103672 HTTP/1.1" 200 None 2026-03-24 00:39:33,589 - lava_callback - INFO - Completed processing callback for node 69c1dccf0efaf21d93103672 INFO: 172.21.0.1:40898 - "GET / HTTP/1.0" 200 OK 2026-03-24 00:39:39,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:39:39,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:44970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56436 - "GET / HTTP/1.0" 200 OK 2026-03-24 00:39:59,953 - lava_callback - INFO - Received callback for node 69c1dcaa0efaf21d93103601 INFO: 172.21.0.1:52872 - "POST /node/69c1dcaa0efaf21d93103601 HTTP/1.0" 202 Accepted 2026-03-24 00:39:59,959 - lava_callback - INFO - Starting async job submit for node 69c1dcaa0efaf21d93103601 2026-03-24 00:39:59,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:39:59,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dcaa0efaf21d93103601 HTTP/1.1" 200 None 2026-03-24 00:39:59,995 - lava_callback - DEBUG - Processing node 69c1dcaa0efaf21d93103601: baseline-arm-clabbe 2026-03-24 00:40:00,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:40:00,360 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:40:00,362 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c1dcaa0efaf21d93103601/log.txt.gz 2026-03-24 00:40:00,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:40:00,679 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:40:00,681 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c1dcaa0efaf21d93103601/lava-logs.yaml 2026-03-24 00:40:00,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:40:00,838 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:40:00,840 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c1dcaa0efaf21d93103601/lava_callback.json.gz 2026-03-24 00:40:00,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:40:00,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dcaa0efaf21d93103601 HTTP/1.1" 200 None 2026-03-24 00:40:00,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:40:00,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-24 00:40:00,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:40:00,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1dcaa0efaf21d93103601 HTTP/1.1" 200 None 2026-03-24 00:40:00,929 - lava_callback - INFO - Completed processing callback for node 69c1dcaa0efaf21d93103601 INFO: 172.21.0.1:52886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52892 - "GET / HTTP/1.0" 200 OK 2026-03-24 00:40:09,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:40:10,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:40614 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55148 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55156 - "GET / HTTP/1.0" 200 OK 2026-03-24 00:41:20,808 - lava_callback - INFO - Received callback for node 69c1dcc80efaf21d93103666 INFO: 172.21.0.1:56720 - "POST /node/69c1dcc80efaf21d93103666 HTTP/1.0" 202 Accepted 2026-03-24 00:41:20,815 - lava_callback - INFO - Starting async job submit for node 69c1dcc80efaf21d93103666 2026-03-24 00:41:20,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:41:20,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dcc80efaf21d93103666 HTTP/1.1" 200 None 2026-03-24 00:41:20,855 - lava_callback - DEBUG - Processing node 69c1dcc80efaf21d93103666: ltp-cap_bounds 2026-03-24 00:41:21,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:41:21,352 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:41:21,354 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69c1dcc80efaf21d93103666/log.txt.gz 2026-03-24 00:41:21,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:41:21,790 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:41:21,792 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69c1dcc80efaf21d93103666/lava-logs.yaml 2026-03-24 00:41:21,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:41:21,954 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:41:21,955 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69c1dcc80efaf21d93103666/lava_callback.json.gz 2026-03-24 00:41:21,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:41:21,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dcc80efaf21d93103666 HTTP/1.1" 200 None 2026-03-24 00:41:22,001 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:41:22,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-24 00:41:22,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:41:22,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1dcc80efaf21d93103666 HTTP/1.1" 200 None 2026-03-24 00:41:22,043 - lava_callback - INFO - Completed processing callback for node 69c1dcc80efaf21d93103666 INFO: 172.21.0.1:56734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57410 - "GET / HTTP/1.0" 200 OK 2026-03-24 00:41:39,070 - lava_callback - INFO - Received callback for node 69c1dcae0efaf21d9310360d INFO: 172.21.0.1:57418 - "POST /node/69c1dcae0efaf21d9310360d HTTP/1.0" 202 Accepted 2026-03-24 00:41:39,076 - lava_callback - INFO - Starting async job submit for node 69c1dcae0efaf21d9310360d 2026-03-24 00:41:39,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:41:39,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dcae0efaf21d9310360d HTTP/1.1" 200 None 2026-03-24 00:41:39,114 - lava_callback - DEBUG - Processing node 69c1dcae0efaf21d9310360d: baseline-arm-clabbe 2026-03-24 00:41:39,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:41:39,476 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:41:39,478 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c1dcae0efaf21d9310360d/log.txt.gz 2026-03-24 00:41:39,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:41:39,776 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:41:39,778 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c1dcae0efaf21d9310360d/lava-logs.yaml 2026-03-24 00:41:39,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:41:39,952 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:41:39,954 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c1dcae0efaf21d9310360d/lava_callback.json.gz 2026-03-24 00:41:39,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:41:39,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dcae0efaf21d9310360d HTTP/1.1" 200 None 2026-03-24 00:41:39,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:41:40,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-24 00:41:40,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:41:40,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:41:40,052 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1dcae0efaf21d9310360d HTTP/1.1" 200 None 2026-03-24 00:41:40,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-24 00:41:40,091 - lava_callback - INFO - Completed processing callback for node 69c1dcae0efaf21d9310360d 2026-03-24 00:41:41,918 - lava_callback - INFO - Received callback for node 69c1dcda0efaf21d931036a3 INFO: 172.21.0.1:33420 - "POST /node/69c1dcda0efaf21d931036a3 HTTP/1.0" 202 Accepted 2026-03-24 00:41:41,925 - lava_callback - INFO - Starting async job submit for node 69c1dcda0efaf21d931036a3 2026-03-24 00:41:41,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:41:41,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dcda0efaf21d931036a3 HTTP/1.1" 200 None 2026-03-24 00:41:41,967 - lava_callback - DEBUG - Processing node 69c1dcda0efaf21d931036a3: ltp-syscalls 2026-03-24 00:41:42,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:41:42,855 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:41:42,857 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-69c1dcda0efaf21d931036a3/log.txt.gz 2026-03-24 00:41:43,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:41:43,491 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:41:43,493 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-69c1dcda0efaf21d931036a3/lava-logs.yaml 2026-03-24 00:41:43,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:41:43,690 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:41:43,692 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-69c1dcda0efaf21d931036a3/lava_callback.json.gz 2026-03-24 00:41:43,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:41:43,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dcda0efaf21d931036a3 HTTP/1.1" 200 None 2026-03-24 00:41:43,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:41:43,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-24 00:41:43,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:41:43,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1dcda0efaf21d931036a3 HTTP/1.1" 200 None 2026-03-24 00:41:43,820 - lava_callback - INFO - Completed processing callback for node 69c1dcda0efaf21d931036a3 INFO: 172.21.0.1:33434 - "GET / HTTP/1.0" 200 OK 2026-03-24 00:41:50,187 - lava_callback - INFO - Received callback for node 69c1dccb0efaf21d9310366c INFO: 172.21.0.1:35940 - "POST /node/69c1dccb0efaf21d9310366c HTTP/1.0" 202 Accepted 2026-03-24 00:41:50,193 - lava_callback - INFO - Starting async job submit for node 69c1dccb0efaf21d9310366c 2026-03-24 00:41:50,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:41:50,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dccb0efaf21d9310366c HTTP/1.1" 200 None 2026-03-24 00:41:50,234 - lava_callback - DEBUG - Processing node 69c1dccb0efaf21d9310366c: ltp-controllers 2026-03-24 00:41:50,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:41:50,803 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:41:50,805 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-controllers-69c1dccb0efaf21d9310366c/log.txt.gz 2026-03-24 00:41:51,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:41:51,277 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:41:51,279 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-controllers-69c1dccb0efaf21d9310366c/lava-logs.yaml 2026-03-24 00:41:51,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:41:51,445 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:41:51,447 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-controllers-69c1dccb0efaf21d9310366c/lava_callback.json.gz 2026-03-24 00:41:51,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:41:51,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dccb0efaf21d9310366c HTTP/1.1" 200 None 2026-03-24 00:41:51,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:41:51,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-24 00:41:51,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:41:51,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1dccb0efaf21d9310366c HTTP/1.1" 200 None 2026-03-24 00:41:51,539 - lava_callback - INFO - Completed processing callback for node 69c1dccb0efaf21d9310366c INFO: 172.21.0.1:35952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46614 - "GET / HTTP/1.0" 200 OK 2026-03-24 00:42:10,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:42:10,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:48200 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51660 - "GET / HTTP/1.0" 200 OK 2026-03-24 00:42:54,941 - lava_callback - INFO - Received callback for node 69bf3b06f1004e82292e342b INFO: 172.21.0.1:34734 - "POST /node/69bf3b06f1004e82292e342b HTTP/1.0" 202 Accepted 2026-03-24 00:42:54,946 - lava_callback - INFO - Starting async job submit for node 69bf3b06f1004e82292e342b 2026-03-24 00:42:54,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:42:54,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3b06f1004e82292e342b HTTP/1.1" 200 None 2026-03-24 00:42:54,963 - lava_callback - DEBUG - Processing node 69bf3b06f1004e82292e342b: baseline-arm-clabbe 2026-03-24 00:42:54,963 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf3b06f1004e82292e342b 2026-03-24 00:42:54,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:42:54,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3b06f1004e82292e342b HTTP/1.1" 200 None 2026-03-24 00:42:54,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:42:54,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-24 00:42:54,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:42:55,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf3b06f1004e82292e342b HTTP/1.1" 200 None 2026-03-24 00:42:55,022 - lava_callback - ERROR - Error processing callback for node 69bf3b06f1004e82292e342b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-24 00:42:55,587 - lava_callback - INFO - Received callback for node 69bf3b0af1004e82292e3438 INFO: 172.21.0.1:34738 - "POST /node/69bf3b0af1004e82292e3438 HTTP/1.0" 202 Accepted 2026-03-24 00:42:55,592 - lava_callback - INFO - Starting async job submit for node 69bf3b0af1004e82292e3438 2026-03-24 00:42:55,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:42:55,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3b0af1004e82292e3438 HTTP/1.1" 200 None 2026-03-24 00:42:55,609 - lava_callback - DEBUG - Processing node 69bf3b0af1004e82292e3438: baseline-arm-clabbe 2026-03-24 00:42:55,609 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf3b0af1004e82292e3438 2026-03-24 00:42:55,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:42:55,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3b0af1004e82292e3438 HTTP/1.1" 200 None 2026-03-24 00:42:55,625 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:42:55,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-24 00:42:55,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:42:55,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf3b0af1004e82292e3438 HTTP/1.1" 200 None 2026-03-24 00:42:55,672 - lava_callback - ERROR - Error processing callback for node 69bf3b0af1004e82292e3438: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-24 00:42:56,228 - lava_callback - INFO - Received callback for node 69bf3b0bf1004e82292e343b INFO: 172.21.0.1:34752 - "POST /node/69bf3b0bf1004e82292e343b HTTP/1.0" 202 Accepted 2026-03-24 00:42:56,233 - lava_callback - INFO - Starting async job submit for node 69bf3b0bf1004e82292e343b 2026-03-24 00:42:56,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:42:56,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3b0bf1004e82292e343b HTTP/1.1" 200 None 2026-03-24 00:42:56,268 - lava_callback - DEBUG - Processing node 69bf3b0bf1004e82292e343b: baseline-arm-clabbe 2026-03-24 00:42:56,268 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf3b0bf1004e82292e343b 2026-03-24 00:42:56,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:42:56,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3b0bf1004e82292e343b HTTP/1.1" 200 None 2026-03-24 00:42:56,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:42:56,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-24 00:42:56,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:42:56,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf3b0bf1004e82292e343b HTTP/1.1" 200 None 2026-03-24 00:42:56,321 - lava_callback - ERROR - Error processing callback for node 69bf3b0bf1004e82292e343b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34758 - "GET / HTTP/1.0" 200 OK 2026-03-24 00:42:56,904 - lava_callback - INFO - Received callback for node 69bf3b0cf1004e82292e343e INFO: 172.21.0.1:34770 - "POST /node/69bf3b0cf1004e82292e343e HTTP/1.0" 202 Accepted 2026-03-24 00:42:56,909 - lava_callback - INFO - Starting async job submit for node 69bf3b0cf1004e82292e343e 2026-03-24 00:42:56,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:42:56,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3b0cf1004e82292e343e HTTP/1.1" 200 None 2026-03-24 00:42:56,926 - lava_callback - DEBUG - Processing node 69bf3b0cf1004e82292e343e: baseline-arm-clabbe 2026-03-24 00:42:56,926 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf3b0cf1004e82292e343e 2026-03-24 00:42:56,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:42:56,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3b0cf1004e82292e343e HTTP/1.1" 200 None 2026-03-24 00:42:56,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:42:56,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-24 00:42:56,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:42:57,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf3b0cf1004e82292e343e HTTP/1.1" 200 None 2026-03-24 00:42:57,022 - lava_callback - ERROR - Error processing callback for node 69bf3b0cf1004e82292e343e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34786 - "GET / HTTP/1.0" 200 OK 2026-03-24 00:43:14,370 - lava_callback - INFO - Received callback for node 69c1de150efaf21d931037a8 INFO: 172.21.0.1:57546 - "POST /node/69c1de150efaf21d931037a8 HTTP/1.0" 202 Accepted 2026-03-24 00:43:14,376 - lava_callback - INFO - Starting async job submit for node 69c1de150efaf21d931037a8 2026-03-24 00:43:14,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:43:14,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1de150efaf21d931037a8 HTTP/1.1" 200 None 2026-03-24 00:43:14,393 - lava_callback - DEBUG - Processing node 69c1de150efaf21d931037a8: baseline-arm 2026-03-24 00:43:14,398 - lava_callback - WARNING - Failed to upload log 2026-03-24 00:43:14,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:43:14,560 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:43:14,562 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c1de150efaf21d931037a8/lava-logs.yaml 2026-03-24 00:43:14,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:43:14,686 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:43:14,687 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c1de150efaf21d931037a8/lava_callback.json.gz 2026-03-24 00:43:14,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:43:14,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1de150efaf21d931037a8 HTTP/1.1" 200 None 2026-03-24 00:43:14,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:43:14,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-24 00:43:14,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:43:14,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1de150efaf21d931037a8 HTTP/1.1" 200 None 2026-03-24 00:43:14,774 - lava_callback - INFO - Completed processing callback for node 69c1de150efaf21d931037a8 2026-03-24 00:43:14,907 - lava_callback - INFO - Received callback for node 69bf3b0ef1004e82292e3448 INFO: 172.21.0.1:57552 - "POST /node/69bf3b0ef1004e82292e3448 HTTP/1.0" 202 Accepted 2026-03-24 00:43:14,912 - lava_callback - INFO - Starting async job submit for node 69bf3b0ef1004e82292e3448 2026-03-24 00:43:14,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:43:14,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3b0ef1004e82292e3448 HTTP/1.1" 200 None 2026-03-24 00:43:14,929 - lava_callback - DEBUG - Processing node 69bf3b0ef1004e82292e3448: baseline-arm-clabbe 2026-03-24 00:43:14,929 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf3b0ef1004e82292e3448 2026-03-24 00:43:14,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:43:14,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3b0ef1004e82292e3448 HTTP/1.1" 200 None 2026-03-24 00:43:14,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:43:14,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-24 00:43:14,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:43:15,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf3b0ef1004e82292e3448 HTTP/1.1" 200 None 2026-03-24 00:43:15,009 - lava_callback - ERROR - Error processing callback for node 69bf3b0ef1004e82292e3448: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57558 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57566 - "GET / HTTP/1.0" 200 OK 2026-03-24 00:43:18,102 - lava_callback - INFO - Received callback for node 69c1dcb00efaf21d93103610 INFO: 172.21.0.1:57574 - "POST /node/69c1dcb00efaf21d93103610 HTTP/1.0" 202 Accepted 2026-03-24 00:43:18,109 - lava_callback - INFO - Starting async job submit for node 69c1dcb00efaf21d93103610 2026-03-24 00:43:18,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:43:18,146 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dcb00efaf21d93103610 HTTP/1.1" 200 None 2026-03-24 00:43:18,147 - lava_callback - DEBUG - Processing node 69c1dcb00efaf21d93103610: baseline-arm-clabbe 2026-03-24 00:43:18,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:43:18,504 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:43:18,506 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c1dcb00efaf21d93103610/log.txt.gz 2026-03-24 00:43:18,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:43:18,917 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:43:18,920 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c1dcb00efaf21d93103610/lava-logs.yaml 2026-03-24 00:43:18,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:43:19,062 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:43:19,064 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c1dcb00efaf21d93103610/lava_callback.json.gz 2026-03-24 00:43:19,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:43:19,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dcb00efaf21d93103610 HTTP/1.1" 200 None 2026-03-24 00:43:19,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:43:19,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-24 00:43:19,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:43:19,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1dcb00efaf21d93103610 HTTP/1.1" 200 None 2026-03-24 00:43:19,143 - lava_callback - INFO - Completed processing callback for node 69c1dcb00efaf21d93103610 INFO: 172.21.0.1:36202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59190 - "GET / HTTP/1.0" 200 OK 2026-03-24 00:43:40,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:43:40,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-24 00:43:40,699 - lava_callback - INFO - Received callback for node 69c1dccd0efaf21d9310366f INFO: 172.21.0.1:40804 - "POST /node/69c1dccd0efaf21d9310366f HTTP/1.0" 202 Accepted 2026-03-24 00:43:40,706 - lava_callback - INFO - Starting async job submit for node 69c1dccd0efaf21d9310366f 2026-03-24 00:43:40,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:43:40,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dccd0efaf21d9310366f HTTP/1.1" 200 None 2026-03-24 00:43:40,749 - lava_callback - DEBUG - Processing node 69c1dccd0efaf21d9310366f: ltp-cve 2026-03-24 00:43:41,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:43:41,334 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:43:41,336 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cve-69c1dccd0efaf21d9310366f/log.txt.gz 2026-03-24 00:43:41,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:43:41,760 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:43:41,762 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cve-69c1dccd0efaf21d9310366f/lava-logs.yaml 2026-03-24 00:43:41,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:43:41,935 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:43:41,937 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cve-69c1dccd0efaf21d9310366f/lava_callback.json.gz 2026-03-24 00:43:41,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:43:41,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dccd0efaf21d9310366f HTTP/1.1" 200 None 2026-03-24 00:43:41,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:43:41,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-24 00:43:41,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:43:42,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1dccd0efaf21d9310366f HTTP/1.1" 200 None 2026-03-24 00:43:42,024 - lava_callback - INFO - Completed processing callback for node 69c1dccd0efaf21d9310366f INFO: 172.21.0.1:40820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42730 - "GET / HTTP/1.0" 200 OK 2026-03-24 00:44:10,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:44:10,149 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:38328 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35004 - "GET / HTTP/1.0" 200 OK 2026-03-24 00:44:36,901 - lava_callback - INFO - Received callback for node 69c1de430efaf21d931037c9 INFO: 172.21.0.1:35018 - "POST /node/69c1de430efaf21d931037c9 HTTP/1.0" 202 Accepted 2026-03-24 00:44:36,907 - lava_callback - INFO - Starting async job submit for node 69c1de430efaf21d931037c9 2026-03-24 00:44:36,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:44:36,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1de430efaf21d931037c9 HTTP/1.1" 200 None 2026-03-24 00:44:36,949 - lava_callback - DEBUG - Processing node 69c1de430efaf21d931037c9: baseline-arm-pengutronix 2026-03-24 00:44:37,107 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:44:37,268 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:44:37,270 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c1de430efaf21d931037c9/log.txt.gz 2026-03-24 00:44:37,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:44:37,569 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:44:37,571 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c1de430efaf21d931037c9/lava-logs.yaml 2026-03-24 00:44:37,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:44:37,751 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:44:37,752 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c1de430efaf21d931037c9/lava_callback.json.gz 2026-03-24 00:44:37,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:44:37,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1de430efaf21d931037c9 HTTP/1.1" 200 None 2026-03-24 00:44:37,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:44:37,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-24 00:44:37,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:44:37,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1de430efaf21d931037c9 HTTP/1.1" 200 None 2026-03-24 00:44:37,832 - lava_callback - INFO - Completed processing callback for node 69c1de430efaf21d931037c9 2026-03-24 00:44:40,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:44:40,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-24 00:44:40,988 - lava_callback - INFO - Received callback for node 69c1dcc90efaf21d93103669 INFO: 172.21.0.1:53738 - "POST /node/69c1dcc90efaf21d93103669 HTTP/1.0" 202 Accepted 2026-03-24 00:44:40,995 - lava_callback - INFO - Starting async job submit for node 69c1dcc90efaf21d93103669 2026-03-24 00:44:41,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:44:41,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dcc90efaf21d93103669 HTTP/1.1" 200 None 2026-03-24 00:44:41,043 - lava_callback - DEBUG - Processing node 69c1dcc90efaf21d93103669: ltp-containers 2026-03-24 00:44:41,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:44:41,624 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:44:41,625 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-containers-69c1dcc90efaf21d93103669/log.txt.gz 2026-03-24 00:44:41,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:44:42,111 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:44:42,113 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-containers-69c1dcc90efaf21d93103669/lava-logs.yaml 2026-03-24 00:44:42,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:44:42,273 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:44:42,275 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-containers-69c1dcc90efaf21d93103669/lava_callback.json.gz 2026-03-24 00:44:42,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:44:42,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dcc90efaf21d93103669 HTTP/1.1" 200 None 2026-03-24 00:44:42,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:44:42,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-24 00:44:42,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:44:42,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1dcc90efaf21d93103669 HTTP/1.1" 200 None 2026-03-24 00:44:42,386 - lava_callback - INFO - Completed processing callback for node 69c1dcc90efaf21d93103669 INFO: 172.21.0.1:53740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36124 - "GET / HTTP/1.0" 200 OK 2026-03-24 00:45:10,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:45:10,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:33532 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33538 - "GET / HTTP/1.0" 200 OK 2026-03-24 00:45:19,633 - lava_callback - INFO - Received callback for node 69c1de680efaf21d931037d4 INFO: 172.21.0.1:33544 - "POST /node/69c1de680efaf21d931037d4 HTTP/1.0" 202 Accepted 2026-03-24 00:45:19,639 - lava_callback - INFO - Starting async job submit for node 69c1de680efaf21d931037d4 2026-03-24 00:45:19,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:45:19,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1de680efaf21d931037d4 HTTP/1.1" 200 None 2026-03-24 00:45:19,672 - lava_callback - DEBUG - Processing node 69c1de680efaf21d931037d4: baseline-arm-pengutronix 2026-03-24 00:45:19,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:45:19,934 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:45:19,936 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c1de680efaf21d931037d4/log.txt.gz 2026-03-24 00:45:20,030 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:45:20,215 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:45:20,217 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c1de680efaf21d931037d4/lava-logs.yaml 2026-03-24 00:45:20,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:45:20,347 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:45:20,348 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c1de680efaf21d931037d4/lava_callback.json.gz 2026-03-24 00:45:20,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:45:20,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1de680efaf21d931037d4 HTTP/1.1" 200 None 2026-03-24 00:45:20,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:45:20,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-24 00:45:20,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:45:20,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1de680efaf21d931037d4 HTTP/1.1" 200 None 2026-03-24 00:45:20,426 - lava_callback - INFO - Completed processing callback for node 69c1de680efaf21d931037d4 2026-03-24 00:45:21,800 - lava_callback - INFO - Received callback for node 69c1dcd00efaf21d93103675 INFO: 172.21.0.1:49262 - "POST /node/69c1dcd00efaf21d93103675 HTTP/1.0" 202 Accepted 2026-03-24 00:45:21,806 - lava_callback - INFO - Starting async job submit for node 69c1dcd00efaf21d93103675 2026-03-24 00:45:21,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:45:21,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dcd00efaf21d93103675 HTTP/1.1" 200 None 2026-03-24 00:45:21,846 - lava_callback - DEBUG - Processing node 69c1dcd00efaf21d93103675: ltp-fs_bind 2026-03-24 00:45:22,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:45:22,369 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:45:22,371 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69c1dcd00efaf21d93103675/log.txt.gz 2026-03-24 00:45:22,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:45:22,795 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:45:22,797 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69c1dcd00efaf21d93103675/lava-logs.yaml 2026-03-24 00:45:22,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:45:22,948 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:45:22,949 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69c1dcd00efaf21d93103675/lava_callback.json.gz 2026-03-24 00:45:22,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:45:22,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dcd00efaf21d93103675 HTTP/1.1" 200 None 2026-03-24 00:45:22,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:45:23,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-24 00:45:23,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:45:23,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1dcd00efaf21d93103675 HTTP/1.1" 200 None 2026-03-24 00:45:23,037 - lava_callback - INFO - Completed processing callback for node 69c1dcd00efaf21d93103675 INFO: 172.21.0.1:49264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35016 - "GET / HTTP/1.0" 200 OK 2026-03-24 00:45:40,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:45:40,210 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:59420 - "GET / HTTP/1.0" 200 OK 2026-03-24 00:45:49,448 - lava_callback - INFO - Received callback for node 69c1dcd20efaf21d93103678 INFO: 172.21.0.1:59424 - "POST /node/69c1dcd20efaf21d93103678 HTTP/1.0" 202 Accepted 2026-03-24 00:45:49,455 - lava_callback - INFO - Starting async job submit for node 69c1dcd20efaf21d93103678 2026-03-24 00:45:49,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:45:49,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dcd20efaf21d93103678 HTTP/1.1" 200 None 2026-03-24 00:45:49,495 - lava_callback - DEBUG - Processing node 69c1dcd20efaf21d93103678: ltp-fsx 2026-03-24 00:45:49,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:45:49,974 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:45:49,976 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fsx-69c1dcd20efaf21d93103678/log.txt.gz 2026-03-24 00:45:50,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:45:50,424 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:45:50,426 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fsx-69c1dcd20efaf21d93103678/lava-logs.yaml 2026-03-24 00:45:50,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:45:50,575 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:45:50,577 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fsx-69c1dcd20efaf21d93103678/lava_callback.json.gz 2026-03-24 00:45:50,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:45:50,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dcd20efaf21d93103678 HTTP/1.1" 200 None 2026-03-24 00:45:50,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:45:50,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-24 00:45:50,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:45:50,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1dcd20efaf21d93103678 HTTP/1.1" 200 None 2026-03-24 00:45:50,667 - lava_callback - INFO - Completed processing callback for node 69c1dcd20efaf21d93103678 INFO: 172.21.0.1:39488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51752 - "GET / HTTP/1.0" 200 OK 2026-03-24 00:46:10,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:46:10,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:35294 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45284 - "GET / HTTP/1.0" 200 OK 2026-03-24 00:46:31,689 - lava_callback - INFO - Received callback for node 69c1da530efaf21d93103588 INFO: 172.21.0.1:37344 - "POST /node/69c1da530efaf21d93103588 HTTP/1.0" 202 Accepted 2026-03-24 00:46:31,696 - lava_callback - INFO - Starting async job submit for node 69c1da530efaf21d93103588 2026-03-24 00:46:31,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:46:31,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1da530efaf21d93103588 HTTP/1.1" 200 None 2026-03-24 00:46:31,774 - lava_callback - DEBUG - Processing node 69c1da530efaf21d93103588: ltp-smoke 2026-03-24 00:46:32,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:46:32,405 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:46:32,407 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c1da530efaf21d93103588/log.txt.gz 2026-03-24 00:46:32,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:46:32,903 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:46:32,905 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c1da530efaf21d93103588/lava-logs.yaml 2026-03-24 00:46:32,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:46:33,104 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:46:33,106 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c1da530efaf21d93103588/lava_callback.json.gz 2026-03-24 00:46:33,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:46:33,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1da530efaf21d93103588 HTTP/1.1" 200 None 2026-03-24 00:46:33,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:46:33,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8340efaf21d93103549 HTTP/1.1" 200 None 2026-03-24 00:46:33,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:46:33,288 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1da530efaf21d93103588 HTTP/1.1" 200 None 2026-03-24 00:46:33,413 - lava_callback - INFO - Completed processing callback for node 69c1da530efaf21d93103588 INFO: 172.21.0.1:37346 - "GET / HTTP/1.0" 200 OK 2026-03-24 00:46:40,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:46:40,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:36818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33064 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34802 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34808 - "GET / HTTP/1.0" 200 OK 2026-03-24 00:48:20,888 - lava_callback - INFO - Received callback for node 69c1dcd30efaf21d9310367b INFO: 172.21.0.1:42328 - "POST /node/69c1dcd30efaf21d9310367b HTTP/1.0" 202 Accepted 2026-03-24 00:48:20,894 - lava_callback - INFO - Starting async job submit for node 69c1dcd30efaf21d9310367b 2026-03-24 00:48:20,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:48:20,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dcd30efaf21d9310367b HTTP/1.1" 200 None 2026-03-24 00:48:20,942 - lava_callback - DEBUG - Processing node 69c1dcd30efaf21d9310367b: ltp-input 2026-03-24 00:48:21,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:48:21,504 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:48:21,506 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-input-69c1dcd30efaf21d9310367b/log.txt.gz 2026-03-24 00:48:21,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:48:21,961 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:48:21,964 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-input-69c1dcd30efaf21d9310367b/lava-logs.yaml 2026-03-24 00:48:21,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:48:22,147 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:48:22,149 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-input-69c1dcd30efaf21d9310367b/lava_callback.json.gz 2026-03-24 00:48:22,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:48:22,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dcd30efaf21d9310367b HTTP/1.1" 200 None 2026-03-24 00:48:22,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:48:22,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-24 00:48:22,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:48:22,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1dcd30efaf21d9310367b HTTP/1.1" 200 None 2026-03-24 00:48:22,252 - lava_callback - INFO - Completed processing callback for node 69c1dcd30efaf21d9310367b INFO: 172.21.0.1:42344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51652 - "GET / HTTP/1.0" 200 OK 2026-03-24 00:48:40,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:48:40,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:36004 - "GET / HTTP/1.0" 200 OK 2026-03-24 00:48:55,336 - lava_callback - INFO - Received callback for node 69bf3c72f1004e82292e3615 INFO: 172.21.0.1:36524 - "POST /node/69bf3c72f1004e82292e3615 HTTP/1.0" 202 Accepted 2026-03-24 00:48:55,342 - lava_callback - INFO - Starting async job submit for node 69bf3c72f1004e82292e3615 2026-03-24 00:48:55,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:48:55,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3c72f1004e82292e3615 HTTP/1.1" 200 None 2026-03-24 00:48:55,359 - lava_callback - DEBUG - Processing node 69bf3c72f1004e82292e3615: baseline-arm-clabbe 2026-03-24 00:48:55,359 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf3c72f1004e82292e3615 2026-03-24 00:48:55,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:48:55,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3c72f1004e82292e3615 HTTP/1.1" 200 None 2026-03-24 00:48:55,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:48:55,389 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92d5b4744551b27f9fbe HTTP/1.1" 200 None 2026-03-24 00:48:55,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:48:55,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf3c72f1004e82292e3615 HTTP/1.1" 200 None 2026-03-24 00:48:55,421 - lava_callback - ERROR - Error processing callback for node 69bf3c72f1004e82292e3615: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55520 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55524 - "GET / HTTP/1.0" 200 OK 2026-03-24 00:49:20,942 - lava_callback - INFO - Received callback for node 69c1dcd60efaf21d93103684 INFO: 172.21.0.1:50122 - "POST /node/69c1dcd60efaf21d93103684 HTTP/1.0" 202 Accepted 2026-03-24 00:49:20,951 - lava_callback - INFO - Starting async job submit for node 69c1dcd60efaf21d93103684 2026-03-24 00:49:20,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:49:20,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dcd60efaf21d93103684 HTTP/1.1" 200 None 2026-03-24 00:49:20,992 - lava_callback - DEBUG - Processing node 69c1dcd60efaf21d93103684: ltp-pty 2026-03-24 00:49:21,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:49:21,474 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:49:21,476 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-pty-69c1dcd60efaf21d93103684/log.txt.gz 2026-03-24 00:49:21,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:49:21,911 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:49:21,913 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-pty-69c1dcd60efaf21d93103684/lava-logs.yaml 2026-03-24 00:49:21,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:49:22,109 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:49:22,111 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-pty-69c1dcd60efaf21d93103684/lava_callback.json.gz 2026-03-24 00:49:22,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:49:22,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dcd60efaf21d93103684 HTTP/1.1" 200 None 2026-03-24 00:49:22,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:49:22,166 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-24 00:49:22,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:49:22,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1dcd60efaf21d93103684 HTTP/1.1" 200 None 2026-03-24 00:49:22,192 - lava_callback - INFO - Completed processing callback for node 69c1dcd60efaf21d93103684 INFO: 172.21.0.1:50126 - "GET / HTTP/1.0" 200 OK 2026-03-24 00:49:35,330 - lava_callback - INFO - Received callback for node 69bf3c98f1004e82292e362b INFO: 172.21.0.1:37678 - "POST /node/69bf3c98f1004e82292e362b HTTP/1.0" 202 Accepted 2026-03-24 00:49:35,335 - lava_callback - INFO - Starting async job submit for node 69bf3c98f1004e82292e362b 2026-03-24 00:49:35,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:49:35,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3c98f1004e82292e362b HTTP/1.1" 200 None 2026-03-24 00:49:35,475 - lava_callback - DEBUG - Processing node 69bf3c98f1004e82292e362b: baseline-arm-clabbe 2026-03-24 00:49:35,475 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf3c98f1004e82292e362b 2026-03-24 00:49:35,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:49:35,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3c98f1004e82292e362b HTTP/1.1" 200 None 2026-03-24 00:49:35,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:49:35,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92d5b4744551b27f9fbe HTTP/1.1" 200 None 2026-03-24 00:49:35,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:49:35,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf3c98f1004e82292e362b HTTP/1.1" 200 None 2026-03-24 00:49:35,561 - lava_callback - ERROR - Error processing callback for node 69bf3c98f1004e82292e362b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37690 - "GET / HTTP/1.0" 200 OK 2026-03-24 00:49:40,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:49:40,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:60668 - "GET / HTTP/1.0" 200 OK 2026-03-24 00:49:50,131 - lava_callback - INFO - Received callback for node 69c1dcd70efaf21d9310369d INFO: 172.21.0.1:34232 - "POST /node/69c1dcd70efaf21d9310369d HTTP/1.0" 202 Accepted 2026-03-24 00:49:50,139 - lava_callback - INFO - Starting async job submit for node 69c1dcd70efaf21d9310369d 2026-03-24 00:49:50,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:49:50,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dcd70efaf21d9310369d HTTP/1.1" 200 None 2026-03-24 00:49:50,196 - lava_callback - DEBUG - Processing node 69c1dcd70efaf21d9310369d: ltp-sched 2026-03-24 00:49:50,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:49:50,959 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:49:50,961 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-sched-69c1dcd70efaf21d9310369d/log.txt.gz 2026-03-24 00:49:51,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:49:51,696 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:49:51,699 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-sched-69c1dcd70efaf21d9310369d/lava-logs.yaml 2026-03-24 00:49:51,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:49:52,199 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:49:52,201 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-sched-69c1dcd70efaf21d9310369d/lava_callback.json.gz 2026-03-24 00:49:52,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:49:52,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dcd70efaf21d9310369d HTTP/1.1" 200 None 2026-03-24 00:49:52,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:49:52,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-24 00:49:52,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:49:52,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1dcd70efaf21d9310369d HTTP/1.1" 200 None 2026-03-24 00:49:52,316 - lava_callback - INFO - Completed processing callback for node 69c1dcd70efaf21d9310369d INFO: 172.21.0.1:34248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33364 - "GET / HTTP/1.0" 200 OK 2026-03-24 00:50:10,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:50:10,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:34474 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59302 - "GET / HTTP/1.0" 200 OK 2026-03-24 00:50:55,429 - lava_callback - INFO - Received callback for node 69bf3ce1f1004e82292e3654 INFO: 172.21.0.1:56678 - "POST /node/69bf3ce1f1004e82292e3654 HTTP/1.0" 202 Accepted 2026-03-24 00:50:55,438 - lava_callback - INFO - Starting async job submit for node 69bf3ce1f1004e82292e3654 2026-03-24 00:50:55,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:50:55,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3ce1f1004e82292e3654 HTTP/1.1" 200 None 2026-03-24 00:50:55,480 - lava_callback - DEBUG - Processing node 69bf3ce1f1004e82292e3654: baseline-arm-clabbe 2026-03-24 00:50:55,480 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf3ce1f1004e82292e3654 2026-03-24 00:50:55,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:50:55,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3ce1f1004e82292e3654 HTTP/1.1" 200 None 2026-03-24 00:50:55,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:50:55,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92d5b4744551b27f9fbe HTTP/1.1" 200 None 2026-03-24 00:50:55,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:50:55,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf3ce1f1004e82292e3654 HTTP/1.1" 200 None 2026-03-24 00:50:55,590 - lava_callback - ERROR - Error processing callback for node 69bf3ce1f1004e82292e3654: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42054 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44878 - "GET / HTTP/1.0" 200 OK 2026-03-24 00:51:35,425 - lava_callback - INFO - Received callback for node 69bf3d08f1004e82292e3670 INFO: 172.21.0.1:60708 - "POST /node/69bf3d08f1004e82292e3670 HTTP/1.0" 202 Accepted 2026-03-24 00:51:35,434 - lava_callback - INFO - Starting async job submit for node 69bf3d08f1004e82292e3670 2026-03-24 00:51:35,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:51:35,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3d08f1004e82292e3670 HTTP/1.1" 200 None 2026-03-24 00:51:35,458 - lava_callback - DEBUG - Processing node 69bf3d08f1004e82292e3670: baseline-arm-clabbe 2026-03-24 00:51:35,458 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf3d08f1004e82292e3670 2026-03-24 00:51:35,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:51:35,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3d08f1004e82292e3670 HTTP/1.1" 200 None 2026-03-24 00:51:35,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:51:35,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92d5b4744551b27f9fbe HTTP/1.1" 200 None 2026-03-24 00:51:35,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:51:35,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf3d08f1004e82292e3670 HTTP/1.1" 200 None 2026-03-24 00:51:35,551 - lava_callback - ERROR - Error processing callback for node 69bf3d08f1004e82292e3670: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58400 - "GET / HTTP/1.0" 200 OK 2026-03-24 00:52:01,031 - lava_callback - INFO - Received callback for node 69c1dcd40efaf21d93103681 INFO: 172.21.0.1:33200 - "POST /node/69c1dcd40efaf21d93103681 HTTP/1.0" 202 Accepted 2026-03-24 00:52:01,050 - lava_callback - INFO - Starting async job submit for node 69c1dcd40efaf21d93103681 2026-03-24 00:52:01,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:52:01,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dcd40efaf21d93103681 HTTP/1.1" 200 None 2026-03-24 00:52:01,153 - lava_callback - DEBUG - Processing node 69c1dcd40efaf21d93103681: ltp-ipc 2026-03-24 00:52:02,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:52:02,345 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:52:02,347 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-ipc-69c1dcd40efaf21d93103681/log.txt.gz 2026-03-24 00:52:02,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:52:03,848 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:52:03,851 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-ipc-69c1dcd40efaf21d93103681/lava-logs.yaml 2026-03-24 00:52:03,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:52:04,700 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:52:04,702 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-ipc-69c1dcd40efaf21d93103681/lava_callback.json.gz 2026-03-24 00:52:04,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:52:04,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dcd40efaf21d93103681 HTTP/1.1" 200 None 2026-03-24 00:52:04,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:52:04,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-24 00:52:04,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:52:04,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1dcd40efaf21d93103681 HTTP/1.1" 200 None 2026-03-24 00:52:04,901 - lava_callback - INFO - Completed processing callback for node 69c1dcd40efaf21d93103681 INFO: 172.21.0.1:33206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33210 - "GET / HTTP/1.0" 200 OK 2026-03-24 00:52:10,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:52:10,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:54146 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-24 00:52:15,472 - lava_callback - INFO - Received callback for node 69bf3d2ef1004e82292e3693 INFO: 172.21.0.1:54158 - "POST /node/69bf3d2ef1004e82292e3693 HTTP/1.0" 202 Accepted 2026-03-24 00:52:15,482 - lava_callback - INFO - Starting async job submit for node 69bf3d2ef1004e82292e3693 2026-03-24 00:52:15,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:52:15,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3d2ef1004e82292e3693 HTTP/1.1" 200 None 2026-03-24 00:52:15,514 - lava_callback - DEBUG - Processing node 69bf3d2ef1004e82292e3693: baseline-arm-clabbe 2026-03-24 00:52:15,515 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf3d2ef1004e82292e3693 2026-03-24 00:52:15,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:52:15,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3d2ef1004e82292e3693 HTTP/1.1" 200 None 2026-03-24 00:52:15,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:52:15,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92d5b4744551b27f9fbe HTTP/1.1" 200 None 2026-03-24 00:52:15,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:52:15,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf3d2ef1004e82292e3693 HTTP/1.1" 200 None 2026-03-24 00:52:15,622 - lava_callback - ERROR - Error processing callback for node 69bf3d2ef1004e82292e3693: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52318 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52320 - "GET / HTTP/1.0" 200 OK 2026-03-24 00:53:21,288 - lava_callback - INFO - Received callback for node 69c1dcdc0efaf21d931036a6 INFO: 172.21.0.1:45898 - "POST /node/69c1dcdc0efaf21d931036a6 HTTP/1.0" 202 Accepted 2026-03-24 00:53:21,297 - lava_callback - INFO - Starting async job submit for node 69c1dcdc0efaf21d931036a6 2026-03-24 00:53:21,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:53:21,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dcdc0efaf21d931036a6 HTTP/1.1" 200 None 2026-03-24 00:53:21,375 - lava_callback - DEBUG - Processing node 69c1dcdc0efaf21d931036a6: ltp-syscalls-ipc 2026-03-24 00:53:21,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:53:23,081 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:53:23,083 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69c1dcdc0efaf21d931036a6/log.txt.gz 2026-03-24 00:53:23,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:53:23,823 - lava_callback - INFO - Received callback for node 69c1dc910efaf21d931035ae INFO: 172.21.0.1:45914 - "POST /node/69c1dc910efaf21d931035ae HTTP/1.0" 202 Accepted 2026-03-24 00:53:23,836 - lava_callback - INFO - Starting async job submit for node 69c1dc910efaf21d931035ae 2026-03-24 00:53:23,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:53:23,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dc910efaf21d931035ae HTTP/1.1" 200 None 2026-03-24 00:53:23,980 - lava_callback - DEBUG - Processing node 69c1dc910efaf21d931035ae: baseline-arm 2026-03-24 00:53:24,769 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:53:24,771 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69c1dcdc0efaf21d931036a6/lava-logs.yaml 2026-03-24 00:53:24,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:53:25,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:53:26,023 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:53:26,025 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69c1dcdc0efaf21d931036a6/lava_callback.json.gz 2026-03-24 00:53:26,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:53:26,093 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:53:26,093 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dcdc0efaf21d931036a6 HTTP/1.1" 200 None 2026-03-24 00:53:26,096 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c1dc910efaf21d931035ae/log.txt.gz 2026-03-24 00:53:26,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:53:26,204 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-24 00:53:26,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:53:26,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1dcdc0efaf21d931036a6 HTTP/1.1" 200 None 2026-03-24 00:53:26,326 - lava_callback - INFO - Completed processing callback for node 69c1dcdc0efaf21d931036a6 INFO: 172.21.0.1:45924 - "GET / HTTP/1.0" 200 OK 2026-03-24 00:53:26,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:53:27,560 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:53:27,562 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c1dc910efaf21d931035ae/lava-logs.yaml 2026-03-24 00:53:27,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:53:28,244 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:53:28,246 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c1dc910efaf21d931035ae/lava_callback.json.gz DEBUG: Setup failed for 69c1dc910efaf21d931035ae. Transitting job node result: incomplete -> fail 2026-03-24 00:53:28,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:53:28,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dc910efaf21d931035ae HTTP/1.1" 200 None 2026-03-24 00:53:28,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:53:28,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-24 00:53:28,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:53:28,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1dc910efaf21d931035ae HTTP/1.1" 200 None 2026-03-24 00:53:28,668 - lava_callback - INFO - Completed processing callback for node 69c1dc910efaf21d931035ae INFO: 172.21.0.1:55282 - "GET / HTTP/1.0" 200 OK 2026-03-24 00:53:40,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:53:40,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:49924 - "GET / HTTP/1.0" 200 OK 2026-03-24 00:53:50,097 - lava_callback - INFO - Received callback for node 69c1dcdd0efaf21d931036a9 INFO: 172.21.0.1:35254 - "POST /node/69c1dcdd0efaf21d931036a9 HTTP/1.0" 202 Accepted 2026-03-24 00:53:50,106 - lava_callback - INFO - Starting async job submit for node 69c1dcdd0efaf21d931036a9 2026-03-24 00:53:50,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:53:50,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dcdd0efaf21d931036a9 HTTP/1.1" 200 None 2026-03-24 00:53:50,170 - lava_callback - DEBUG - Processing node 69c1dcdd0efaf21d931036a9: ltp-timers 2026-03-24 00:53:50,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:53:51,336 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:53:51,338 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-timers-69c1dcdd0efaf21d931036a9/log.txt.gz 2026-03-24 00:53:51,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:53:52,298 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:53:52,300 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-timers-69c1dcdd0efaf21d931036a9/lava-logs.yaml 2026-03-24 00:53:52,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:53:52,892 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:53:52,894 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-timers-69c1dcdd0efaf21d931036a9/lava_callback.json.gz 2026-03-24 00:53:52,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:53:52,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dcdd0efaf21d931036a9 HTTP/1.1" 200 None 2026-03-24 00:53:52,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:53:52,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-24 00:53:52,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:53:53,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1dcdd0efaf21d931036a9 HTTP/1.1" 200 None 2026-03-24 00:53:53,035 - lava_callback - INFO - Completed processing callback for node 69c1dcdd0efaf21d931036a9 INFO: 172.21.0.1:35260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44342 - "GET / HTTP/1.0" 200 OK 2026-03-24 00:54:10,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:54:10,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:33830 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51502 - "GET / HTTP/1.0" 200 OK 2026-03-24 00:55:01,384 - lava_callback - INFO - Received callback for node 69c1dcde0efaf21d931036ac INFO: 172.21.0.1:40570 - "POST /node/69c1dcde0efaf21d931036ac HTTP/1.0" 202 Accepted 2026-03-24 00:55:01,394 - lava_callback - INFO - Starting async job submit for node 69c1dcde0efaf21d931036ac 2026-03-24 00:55:01,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:55:01,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dcde0efaf21d931036ac HTTP/1.1" 200 None 2026-03-24 00:55:01,460 - lava_callback - DEBUG - Processing node 69c1dcde0efaf21d931036ac: ltp-watchqueue 2026-03-24 00:55:02,054 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:55:03,047 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:55:03,050 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69c1dcde0efaf21d931036ac/log.txt.gz 2026-03-24 00:55:03,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:55:04,402 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:55:04,404 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69c1dcde0efaf21d931036ac/lava-logs.yaml 2026-03-24 00:55:04,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:55:05,399 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:55:05,401 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69c1dcde0efaf21d931036ac/lava_callback.json.gz 2026-03-24 00:55:05,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:55:05,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dcde0efaf21d931036ac HTTP/1.1" 200 None 2026-03-24 00:55:05,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:55:05,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-24 00:55:05,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:55:05,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1dcde0efaf21d931036ac HTTP/1.1" 200 None 2026-03-24 00:55:05,563 - lava_callback - INFO - Completed processing callback for node 69c1dcde0efaf21d931036ac INFO: 172.21.0.1:40582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40592 - "GET / HTTP/1.0" 200 OK 2026-03-24 00:55:10,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:55:10,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:49494 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38288 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35480 - "GET / HTTP/1.0" 200 OK 2026-03-24 00:56:39,414 - lava_callback - INFO - Received callback for node 69c1dcb70efaf21d9310362d INFO: 172.21.0.1:35494 - "POST /node/69c1dcb70efaf21d9310362d HTTP/1.0" 202 Accepted 2026-03-24 00:56:39,461 - lava_callback - INFO - Starting async job submit for node 69c1dcb70efaf21d9310362d 2026-03-24 00:56:39,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:56:39,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dcb70efaf21d9310362d HTTP/1.1" 200 None 2026-03-24 00:56:39,502 - lava_callback - DEBUG - Processing node 69c1dcb70efaf21d9310362d: baseline-arm-pengutronix 2026-03-24 00:56:39,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:56:39,718 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:56:39,720 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c1dcb70efaf21d9310362d/log.txt.gz 2026-03-24 00:56:39,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:56:39,953 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:56:39,955 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c1dcb70efaf21d9310362d/lava-logs.yaml 2026-03-24 00:56:39,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 00:56:40,095 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 00:56:40,096 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c1dcb70efaf21d9310362d/lava_callback.json.gz DEBUG: Setup failed for 69c1dcb70efaf21d9310362d. Transitting job node result: incomplete -> fail 2026-03-24 00:56:40,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:56:40,166 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dcb70efaf21d9310362d HTTP/1.1" 200 None 2026-03-24 00:56:40,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:56:40,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-24 00:56:40,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:56:40,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1dcb70efaf21d9310362d HTTP/1.1" 200 None 2026-03-24 00:56:40,259 - lava_callback - INFO - Completed processing callback for node 69c1dcb70efaf21d9310362d 2026-03-24 00:56:40,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:56:40,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:42078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33684 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51154 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41906 - "GET / HTTP/1.0" 200 OK 2026-03-24 00:59:14,092 - lava_callback - INFO - Received callback for node 69bf3edcf1004e82292e3789 INFO: 172.21.0.1:45840 - "POST /node/69bf3edcf1004e82292e3789 HTTP/1.0" 202 Accepted 2026-03-24 00:59:14,097 - lava_callback - INFO - Starting async job submit for node 69bf3edcf1004e82292e3789 2026-03-24 00:59:14,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:59:14,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3edcf1004e82292e3789 HTTP/1.1" 200 None 2026-03-24 00:59:14,122 - lava_callback - DEBUG - Processing node 69bf3edcf1004e82292e3789: baseline-arm-baylibre 2026-03-24 00:59:14,122 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf3edcf1004e82292e3789 2026-03-24 00:59:14,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:59:14,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3edcf1004e82292e3789 HTTP/1.1" 200 None 2026-03-24 00:59:14,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:59:14,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef76d5731cbd74145fb9 HTTP/1.1" 200 None 2026-03-24 00:59:14,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:59:14,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf3edcf1004e82292e3789 HTTP/1.1" 200 None 2026-03-24 00:59:14,184 - lava_callback - ERROR - Error processing callback for node 69bf3edcf1004e82292e3789: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45850 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50212 - "GET / HTTP/1.0" 200 OK 2026-03-24 00:59:53,887 - lava_callback - INFO - Received callback for node 69bf3f00f1004e82292e3799 INFO: 172.21.0.1:51496 - "POST /node/69bf3f00f1004e82292e3799 HTTP/1.0" 202 Accepted 2026-03-24 00:59:53,893 - lava_callback - INFO - Starting async job submit for node 69bf3f00f1004e82292e3799 2026-03-24 00:59:53,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:59:53,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3f00f1004e82292e3799 HTTP/1.1" 200 None 2026-03-24 00:59:53,909 - lava_callback - DEBUG - Processing node 69bf3f00f1004e82292e3799: baseline-arm-baylibre 2026-03-24 00:59:53,909 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf3f00f1004e82292e3799 2026-03-24 00:59:53,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:59:53,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3f00f1004e82292e3799 HTTP/1.1" 200 None 2026-03-24 00:59:53,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:59:53,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef76d5731cbd74145fb9 HTTP/1.1" 200 None 2026-03-24 00:59:53,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 00:59:53,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf3f00f1004e82292e3799 HTTP/1.1" 200 None 2026-03-24 00:59:53,984 - lava_callback - ERROR - Error processing callback for node 69bf3f00f1004e82292e3799: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56248 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50282 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:00:33,995 - lava_callback - INFO - Received callback for node 69bf3f25f1004e82292e37b0 INFO: 172.21.0.1:54004 - "POST /node/69bf3f25f1004e82292e37b0 HTTP/1.0" 202 Accepted 2026-03-24 01:00:34,004 - lava_callback - INFO - Starting async job submit for node 69bf3f25f1004e82292e37b0 2026-03-24 01:00:34,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:00:34,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3f25f1004e82292e37b0 HTTP/1.1" 200 None 2026-03-24 01:00:34,026 - lava_callback - DEBUG - Processing node 69bf3f25f1004e82292e37b0: baseline-arm-baylibre 2026-03-24 01:00:34,027 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf3f25f1004e82292e37b0 2026-03-24 01:00:34,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:00:34,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3f25f1004e82292e37b0 HTTP/1.1" 200 None 2026-03-24 01:00:34,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:00:34,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef76d5731cbd74145fb9 HTTP/1.1" 200 None 2026-03-24 01:00:34,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:00:34,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf3f25f1004e82292e37b0 HTTP/1.1" 200 None 2026-03-24 01:00:34,096 - lava_callback - ERROR - Error processing callback for node 69bf3f25f1004e82292e37b0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43166 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51154 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:01:54,067 - lava_callback - INFO - Received callback for node 69bf3f6df1004e82292e37d1 INFO: 172.21.0.1:60366 - "POST /node/69bf3f6df1004e82292e37d1 HTTP/1.0" 202 Accepted 2026-03-24 01:01:54,072 - lava_callback - INFO - Starting async job submit for node 69bf3f6df1004e82292e37d1 2026-03-24 01:01:54,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:01:54,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3f6df1004e82292e37d1 HTTP/1.1" 200 None 2026-03-24 01:01:54,121 - lava_callback - DEBUG - Processing node 69bf3f6df1004e82292e37d1: baseline-arm-baylibre 2026-03-24 01:01:54,121 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf3f6df1004e82292e37d1 2026-03-24 01:01:54,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:01:54,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3f6df1004e82292e37d1 HTTP/1.1" 200 None 2026-03-24 01:01:54,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:01:54,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef76d5731cbd74145fb9 HTTP/1.1" 200 None 2026-03-24 01:01:54,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:01:54,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf3f6df1004e82292e37d1 HTTP/1.1" 200 None 2026-03-24 01:01:54,185 - lava_callback - ERROR - Error processing callback for node 69bf3f6df1004e82292e37d1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43862 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:02:14,046 - lava_callback - INFO - Received callback for node 69bf3f92f1004e82292e37df INFO: 172.21.0.1:41590 - "POST /node/69bf3f92f1004e82292e37df HTTP/1.0" 202 Accepted 2026-03-24 01:02:14,052 - lava_callback - INFO - Starting async job submit for node 69bf3f92f1004e82292e37df 2026-03-24 01:02:14,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:02:14,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3f92f1004e82292e37df HTTP/1.1" 200 None 2026-03-24 01:02:14,078 - lava_callback - DEBUG - Processing node 69bf3f92f1004e82292e37df: baseline-arm-baylibre 2026-03-24 01:02:14,078 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf3f92f1004e82292e37df 2026-03-24 01:02:14,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:02:14,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3f92f1004e82292e37df HTTP/1.1" 200 None 2026-03-24 01:02:14,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:02:14,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef76d5731cbd74145fb9 HTTP/1.1" 200 None 2026-03-24 01:02:14,107 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:02:14,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf3f92f1004e82292e37df HTTP/1.1" 200 None 2026-03-24 01:02:14,135 - lava_callback - ERROR - Error processing callback for node 69bf3f92f1004e82292e37df: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41592 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59404 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:02:54,058 - lava_callback - INFO - Received callback for node 69bf3fb5f1004e82292e37f1 INFO: 172.21.0.1:36590 - "POST /node/69bf3fb5f1004e82292e37f1 HTTP/1.0" 202 Accepted 2026-03-24 01:02:54,064 - lava_callback - INFO - Starting async job submit for node 69bf3fb5f1004e82292e37f1 2026-03-24 01:02:54,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:02:54,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3fb5f1004e82292e37f1 HTTP/1.1" 200 None 2026-03-24 01:02:54,080 - lava_callback - DEBUG - Processing node 69bf3fb5f1004e82292e37f1: baseline-arm-baylibre 2026-03-24 01:02:54,081 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf3fb5f1004e82292e37f1 2026-03-24 01:02:54,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:02:54,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3fb5f1004e82292e37f1 HTTP/1.1" 200 None 2026-03-24 01:02:54,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:02:54,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef76d5731cbd74145fb9 HTTP/1.1" 200 None 2026-03-24 01:02:54,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:02:54,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf3fb5f1004e82292e37f1 HTTP/1.1" 200 None 2026-03-24 01:02:54,134 - lava_callback - ERROR - Error processing callback for node 69bf3fb5f1004e82292e37f1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36886 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43516 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-24 01:04:16,005 - lava_callback - INFO - Received callback for node 69bf3ffef1004e82292e3815 INFO: 172.21.0.1:43520 - "POST /node/69bf3ffef1004e82292e3815 HTTP/1.0" 202 Accepted 2026-03-24 01:04:16,011 - lava_callback - INFO - Starting async job submit for node 69bf3ffef1004e82292e3815 2026-03-24 01:04:16,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:04:16,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3ffef1004e82292e3815 HTTP/1.1" 200 None 2026-03-24 01:04:16,027 - lava_callback - DEBUG - Processing node 69bf3ffef1004e82292e3815: baseline-arm-clabbe 2026-03-24 01:04:16,028 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf3ffef1004e82292e3815 2026-03-24 01:04:16,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:04:16,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3ffef1004e82292e3815 HTTP/1.1" 200 None 2026-03-24 01:04:16,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:04:16,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef76d5731cbd74145fb9 HTTP/1.1" 200 None 2026-03-24 01:04:16,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:04:16,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf3ffef1004e82292e3815 HTTP/1.1" 200 None 2026-03-24 01:04:16,083 - lava_callback - ERROR - Error processing callback for node 69bf3ffef1004e82292e3815: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54992 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:04:35,957 - lava_callback - INFO - Received callback for node 69bf4021f1004e82292e382a INFO: 172.21.0.1:47054 - "POST /node/69bf4021f1004e82292e382a HTTP/1.0" 202 Accepted 2026-03-24 01:04:35,966 - lava_callback - INFO - Starting async job submit for node 69bf4021f1004e82292e382a 2026-03-24 01:04:35,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:04:35,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf4021f1004e82292e382a HTTP/1.1" 200 None 2026-03-24 01:04:35,989 - lava_callback - DEBUG - Processing node 69bf4021f1004e82292e382a: baseline-arm-clabbe 2026-03-24 01:04:35,989 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf4021f1004e82292e382a 2026-03-24 01:04:35,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:04:36,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf4021f1004e82292e382a HTTP/1.1" 200 None 2026-03-24 01:04:36,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:04:36,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef76d5731cbd74145fb9 HTTP/1.1" 200 None 2026-03-24 01:04:36,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:04:36,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf4021f1004e82292e382a HTTP/1.1" 200 None 2026-03-24 01:04:36,054 - lava_callback - ERROR - Error processing callback for node 69bf4021f1004e82292e382a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39094 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-24 01:05:15,953 - lava_callback - INFO - Received callback for node 69bf4046f1004e82292e3835 INFO: 172.21.0.1:39104 - "POST /node/69bf4046f1004e82292e3835 HTTP/1.0" 202 Accepted 2026-03-24 01:05:15,958 - lava_callback - INFO - Starting async job submit for node 69bf4046f1004e82292e3835 2026-03-24 01:05:15,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:05:15,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf4046f1004e82292e3835 HTTP/1.1" 200 None 2026-03-24 01:05:15,980 - lava_callback - DEBUG - Processing node 69bf4046f1004e82292e3835: baseline-arm-clabbe 2026-03-24 01:05:15,980 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf4046f1004e82292e3835 2026-03-24 01:05:15,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:05:16,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf4046f1004e82292e3835 HTTP/1.1" 200 None 2026-03-24 01:05:16,016 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:05:16,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef76d5731cbd74145fb9 HTTP/1.1" 200 None 2026-03-24 01:05:16,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:05:16,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf4046f1004e82292e3835 HTTP/1.1" 200 None 2026-03-24 01:05:16,089 - lava_callback - ERROR - Error processing callback for node 69bf4046f1004e82292e3835: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57500 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:05:56,025 - lava_callback - INFO - Received callback for node 69bf406af1004e82292e3846 INFO: 172.21.0.1:46468 - "POST /node/69bf406af1004e82292e3846 HTTP/1.0" 202 Accepted 2026-03-24 01:05:56,031 - lava_callback - INFO - Starting async job submit for node 69bf406af1004e82292e3846 2026-03-24 01:05:56,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:05:56,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf406af1004e82292e3846 HTTP/1.1" 200 None 2026-03-24 01:05:56,046 - lava_callback - DEBUG - Processing node 69bf406af1004e82292e3846: baseline-arm-clabbe 2026-03-24 01:05:56,047 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf406af1004e82292e3846 2026-03-24 01:05:56,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:05:56,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf406af1004e82292e3846 HTTP/1.1" 200 None 2026-03-24 01:05:56,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:05:56,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef76d5731cbd74145fb9 HTTP/1.1" 200 None 2026-03-24 01:05:56,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:05:56,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf406af1004e82292e3846 HTTP/1.1" 200 None 2026-03-24 01:05:56,125 - lava_callback - ERROR - Error processing callback for node 69bf406af1004e82292e3846: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42082 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42098 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:06:19,302 - lava_callback - INFO - Received callback for node 69c1e3800efaf21d93105fd6 INFO: 172.21.0.1:42114 - "POST /node/69c1e3800efaf21d93105fd6 HTTP/1.0" 202 Accepted 2026-03-24 01:06:19,308 - lava_callback - INFO - Starting async job submit for node 69c1e3800efaf21d93105fd6 2026-03-24 01:06:19,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:06:19,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3800efaf21d93105fd6 HTTP/1.1" 200 None 2026-03-24 01:06:19,327 - lava_callback - DEBUG - Processing node 69c1e3800efaf21d93105fd6: baseline-arm64 2026-03-24 01:06:19,332 - lava_callback - WARNING - Failed to upload log 2026-03-24 01:06:19,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:06:19,466 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:06:19,467 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1e3800efaf21d93105fd6/lava-logs.yaml 2026-03-24 01:06:19,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:06:19,596 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:06:19,598 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1e3800efaf21d93105fd6/lava_callback.json.gz 2026-03-24 01:06:19,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:06:19,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3800efaf21d93105fd6 HTTP/1.1" 200 None 2026-03-24 01:06:19,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:06:19,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-24 01:06:19,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:06:19,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e3800efaf21d93105fd6 HTTP/1.1" 200 None 2026-03-24 01:06:19,662 - lava_callback - INFO - Completed processing callback for node 69c1e3800efaf21d93105fd6 INFO: 172.21.0.1:48562 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:06:36,090 - lava_callback - INFO - Received callback for node 69bf408ff1004e82292e3857 INFO: 172.21.0.1:41208 - "POST /node/69bf408ff1004e82292e3857 HTTP/1.0" 202 Accepted 2026-03-24 01:06:36,096 - lava_callback - INFO - Starting async job submit for node 69bf408ff1004e82292e3857 2026-03-24 01:06:36,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:06:36,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf408ff1004e82292e3857 HTTP/1.1" 200 None 2026-03-24 01:06:36,112 - lava_callback - DEBUG - Processing node 69bf408ff1004e82292e3857: baseline-arm-clabbe 2026-03-24 01:06:36,112 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf408ff1004e82292e3857 2026-03-24 01:06:36,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:06:36,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf408ff1004e82292e3857 HTTP/1.1" 200 None 2026-03-24 01:06:36,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:06:36,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef76d5731cbd74145fb9 HTTP/1.1" 200 None 2026-03-24 01:06:36,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:06:36,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf408ff1004e82292e3857 HTTP/1.1" 200 None 2026-03-24 01:06:36,169 - lava_callback - ERROR - Error processing callback for node 69bf408ff1004e82292e3857: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41210 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:06:40,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:06:40,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:53028 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:06:50,065 - lava_callback - INFO - Received callback for node 69c1e3a00efaf21d93106037 INFO: 172.21.0.1:39804 - "POST /node/69c1e3a00efaf21d93106037 HTTP/1.0" 202 Accepted 2026-03-24 01:06:50,071 - lava_callback - INFO - Starting async job submit for node 69c1e3a00efaf21d93106037 2026-03-24 01:06:50,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:06:50,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3a00efaf21d93106037 HTTP/1.1" 200 None 2026-03-24 01:06:50,090 - lava_callback - DEBUG - Processing node 69c1e3a00efaf21d93106037: baseline-nfs-arm64 2026-03-24 01:06:50,091 - lava_callback - WARNING - Failed to upload log 2026-03-24 01:06:50,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:06:50,227 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:06:50,229 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c1e3a00efaf21d93106037/lava-logs.yaml 2026-03-24 01:06:50,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:06:50,529 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:06:50,531 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c1e3a00efaf21d93106037/lava_callback.json.gz 2026-03-24 01:06:50,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:06:50,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3a00efaf21d93106037 HTTP/1.1" 200 None 2026-03-24 01:06:50,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:06:50,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-24 01:06:50,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:06:50,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e3a00efaf21d93106037 HTTP/1.1" 200 None 2026-03-24 01:06:50,613 - lava_callback - INFO - Completed processing callback for node 69c1e3a00efaf21d93106037 INFO: 172.21.0.1:39814 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:07:00,834 - lava_callback - INFO - Received callback for node 69c1e38c0efaf21d9310600b INFO: 172.21.0.1:53948 - "POST /node/69c1e38c0efaf21d9310600b HTTP/1.0" 202 Accepted 2026-03-24 01:07:00,841 - lava_callback - INFO - Starting async job submit for node 69c1e38c0efaf21d9310600b 2026-03-24 01:07:00,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:07:00,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e38c0efaf21d9310600b HTTP/1.1" 200 None 2026-03-24 01:07:00,882 - lava_callback - DEBUG - Processing node 69c1e38c0efaf21d9310600b: baseline-arm64 2026-03-24 01:07:01,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:07:01,163 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:07:01,165 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1e38c0efaf21d9310600b/log.txt.gz 2026-03-24 01:07:01,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:07:01,520 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:07:01,522 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1e38c0efaf21d9310600b/lava-logs.yaml 2026-03-24 01:07:01,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:07:01,668 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:07:01,670 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1e38c0efaf21d9310600b/lava_callback.json.gz 2026-03-24 01:07:01,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:07:01,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e38c0efaf21d9310600b HTTP/1.1" 200 None 2026-03-24 01:07:01,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:07:01,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-24 01:07:01,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:07:01,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e38c0efaf21d9310600b HTTP/1.1" 200 None 2026-03-24 01:07:01,750 - lava_callback - INFO - Completed processing callback for node 69c1e38c0efaf21d9310600b INFO: 172.21.0.1:53958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53974 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:07:10,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:07:10,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-24 01:07:11,483 - lava_callback - INFO - Received callback for node 69c1e3810efaf21d93105fdc INFO: 172.21.0.1:34080 - "POST /node/69c1e3810efaf21d93105fdc HTTP/1.0" 202 Accepted 2026-03-24 01:07:11,489 - lava_callback - INFO - Starting async job submit for node 69c1e3810efaf21d93105fdc 2026-03-24 01:07:11,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:07:11,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3810efaf21d93105fdc HTTP/1.1" 200 None 2026-03-24 01:07:11,527 - lava_callback - DEBUG - Processing node 69c1e3810efaf21d93105fdc: baseline-arm64 2026-03-24 01:07:11,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:07:12,000 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:07:12,002 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1e3810efaf21d93105fdc/log.txt.gz 2026-03-24 01:07:12,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:07:12,506 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:07:12,508 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1e3810efaf21d93105fdc/lava-logs.yaml 2026-03-24 01:07:12,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:07:12,638 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:07:12,639 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1e3810efaf21d93105fdc/lava_callback.json.gz 2026-03-24 01:07:12,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:07:12,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3810efaf21d93105fdc HTTP/1.1" 200 None 2026-03-24 01:07:12,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:07:12,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-24 01:07:12,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:07:12,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e3810efaf21d93105fdc HTTP/1.1" 200 None 2026-03-24 01:07:12,735 - lava_callback - INFO - Completed processing callback for node 69c1e3810efaf21d93105fdc INFO: 172.21.0.1:34094 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34106 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:07:19,940 - lava_callback - INFO - Received callback for node 69c1e3830efaf21d93105fe2 INFO: 172.21.0.1:43084 - "POST /node/69c1e3830efaf21d93105fe2 HTTP/1.0" 202 Accepted 2026-03-24 01:07:19,947 - lava_callback - INFO - Starting async job submit for node 69c1e3830efaf21d93105fe2 2026-03-24 01:07:19,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:07:19,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3830efaf21d93105fe2 HTTP/1.1" 200 None 2026-03-24 01:07:19,998 - lava_callback - DEBUG - Processing node 69c1e3830efaf21d93105fe2: baseline-arm64 2026-03-24 01:07:20,145 - lava_callback - WARNING - Failed to upload log 2026-03-24 01:07:20,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:07:20,547 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:07:20,549 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1e3830efaf21d93105fe2/lava-logs.yaml 2026-03-24 01:07:20,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:07:20,666 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:07:20,667 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1e3830efaf21d93105fe2/lava_callback.json.gz 2026-03-24 01:07:20,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:07:20,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3830efaf21d93105fe2 HTTP/1.1" 200 None 2026-03-24 01:07:20,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:07:20,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-24 01:07:20,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:07:20,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e3830efaf21d93105fe2 HTTP/1.1" 200 None 2026-03-24 01:07:20,813 - lava_callback - INFO - Completed processing callback for node 69c1e3830efaf21d93105fe2 INFO: 172.21.0.1:43092 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:07:29,454 - lava_callback - INFO - Received callback for node 69c1e3910efaf21d93106017 INFO: 172.21.0.1:43096 - "POST /node/69c1e3910efaf21d93106017 HTTP/1.0" 202 Accepted 2026-03-24 01:07:29,461 - lava_callback - INFO - Starting async job submit for node 69c1e3910efaf21d93106017 2026-03-24 01:07:29,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:07:29,486 - lava_callback - INFO - Received callback for node 69c1e3b60efaf21d9310608e INFO: 172.21.0.1:43108 - "POST /node/69c1e3b60efaf21d9310608e HTTP/1.0" 202 Accepted 2026-03-24 01:07:29,494 - lava_callback - INFO - Starting async job submit for node 69c1e3b60efaf21d9310608e 2026-03-24 01:07:29,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:07:29,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3910efaf21d93106017 HTTP/1.1" 200 None 2026-03-24 01:07:29,522 - lava_callback - DEBUG - Processing node 69c1e3910efaf21d93106017: baseline-arm64 2026-03-24 01:07:29,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3b60efaf21d9310608e HTTP/1.1" 200 None 2026-03-24 01:07:29,625 - lava_callback - DEBUG - Processing node 69c1e3b60efaf21d9310608e: baseline-arm64 2026-03-24 01:07:30,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:07:30,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:07:30,222 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:07:30,224 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1e3910efaf21d93106017/log.txt.gz 2026-03-24 01:07:30,255 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:07:30,277 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1e3b60efaf21d9310608e/log.txt.gz 2026-03-24 01:07:30,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:07:30,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:07:30,648 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:07:30,650 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1e3b60efaf21d9310608e/lava-logs.yaml 2026-03-24 01:07:30,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:07:30,702 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:07:30,704 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1e3910efaf21d93106017/lava-logs.yaml 2026-03-24 01:07:30,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:07:30,788 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:07:30,789 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1e3b60efaf21d9310608e/lava_callback.json.gz 2026-03-24 01:07:30,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:07:30,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3b60efaf21d9310608e HTTP/1.1" 200 None 2026-03-24 01:07:30,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:07:30,837 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:07:30,839 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1e3910efaf21d93106017/lava_callback.json.gz 2026-03-24 01:07:30,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-24 01:07:30,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:07:30,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:07:30,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3910efaf21d93106017 HTTP/1.1" 200 None 2026-03-24 01:07:30,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:07:30,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e3b60efaf21d9310608e HTTP/1.1" 200 None 2026-03-24 01:07:30,902 - lava_callback - INFO - Completed processing callback for node 69c1e3b60efaf21d9310608e 2026-03-24 01:07:30,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-24 01:07:30,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:07:30,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e3910efaf21d93106017 HTTP/1.1" 200 None 2026-03-24 01:07:30,982 - lava_callback - INFO - Completed processing callback for node 69c1e3910efaf21d93106017 2026-03-24 01:07:32,022 - lava_callback - INFO - Received callback for node 69c1e38d0efaf21d9310600e INFO: 172.21.0.1:38460 - "POST /node/69c1e38d0efaf21d9310600e HTTP/1.0" 202 Accepted 2026-03-24 01:07:32,028 - lava_callback - INFO - Starting async job submit for node 69c1e38d0efaf21d9310600e 2026-03-24 01:07:32,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:07:32,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e38d0efaf21d9310600e HTTP/1.1" 200 None 2026-03-24 01:07:32,065 - lava_callback - DEBUG - Processing node 69c1e38d0efaf21d9310600e: baseline-arm64 2026-03-24 01:07:32,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:07:32,557 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:07:32,559 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1e38d0efaf21d9310600e/log.txt.gz 2026-03-24 01:07:32,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:07:32,954 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:07:32,956 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1e38d0efaf21d9310600e/lava-logs.yaml 2026-03-24 01:07:32,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:07:33,212 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:07:33,214 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1e38d0efaf21d9310600e/lava_callback.json.gz 2026-03-24 01:07:33,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:07:33,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e38d0efaf21d9310600e HTTP/1.1" 200 None 2026-03-24 01:07:33,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:07:33,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-24 01:07:33,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:07:33,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e38d0efaf21d9310600e HTTP/1.1" 200 None 2026-03-24 01:07:33,293 - lava_callback - INFO - Completed processing callback for node 69c1e38d0efaf21d9310600e 2026-03-24 01:07:35,700 - lava_callback - INFO - Received callback for node 69c1e3890efaf21d93106003 INFO: 172.21.0.1:38466 - "POST /node/69c1e3890efaf21d93106003 HTTP/1.0" 202 Accepted 2026-03-24 01:07:35,707 - lava_callback - INFO - Starting async job submit for node 69c1e3890efaf21d93106003 2026-03-24 01:07:35,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:07:35,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3890efaf21d93106003 HTTP/1.1" 200 None 2026-03-24 01:07:35,750 - lava_callback - DEBUG - Processing node 69c1e3890efaf21d93106003: baseline-arm64 2026-03-24 01:07:36,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:07:36,302 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:07:36,304 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1e3890efaf21d93106003/log.txt.gz INFO: 172.21.0.1:38474 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:07:36,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:07:36,751 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:07:36,753 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1e3890efaf21d93106003/lava-logs.yaml 2026-03-24 01:07:36,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:07:36,923 - lava_callback - INFO - Received callback for node 69c1e3880efaf21d93106000 INFO: 172.21.0.1:38484 - "POST /node/69c1e3880efaf21d93106000 HTTP/1.0" 202 Accepted 2026-03-24 01:07:36,930 - lava_callback - INFO - Starting async job submit for node 69c1e3880efaf21d93106000 2026-03-24 01:07:36,948 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:07:36,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:07:36,955 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1e3890efaf21d93106003/lava_callback.json.gz 2026-03-24 01:07:36,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:07:36,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3880efaf21d93106000 HTTP/1.1" 200 None 2026-03-24 01:07:36,996 - lava_callback - DEBUG - Processing node 69c1e3880efaf21d93106000: baseline-arm64 2026-03-24 01:07:37,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3890efaf21d93106003 HTTP/1.1" 200 None 2026-03-24 01:07:37,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:07:37,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-24 01:07:37,112 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:07:37,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e3890efaf21d93106003 HTTP/1.1" 200 None 2026-03-24 01:07:37,291 - lava_callback - INFO - Completed processing callback for node 69c1e3890efaf21d93106003 2026-03-24 01:07:37,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:07:37,601 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:07:37,603 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1e3880efaf21d93106000/log.txt.gz 2026-03-24 01:07:37,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:07:38,076 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:07:38,078 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1e3880efaf21d93106000/lava-logs.yaml 2026-03-24 01:07:38,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:07:38,211 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:07:38,213 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1e3880efaf21d93106000/lava_callback.json.gz 2026-03-24 01:07:38,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:07:38,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3880efaf21d93106000 HTTP/1.1" 200 None 2026-03-24 01:07:38,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:07:38,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-24 01:07:38,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:07:38,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e3880efaf21d93106000 HTTP/1.1" 200 None 2026-03-24 01:07:38,304 - lava_callback - INFO - Completed processing callback for node 69c1e3880efaf21d93106000 2026-03-24 01:07:40,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:07:40,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-24 01:07:45,295 - lava_callback - INFO - Received callback for node 69c1e38f0efaf21d93106014 INFO: 172.21.0.1:54650 - "POST /node/69c1e38f0efaf21d93106014 HTTP/1.0" 202 Accepted 2026-03-24 01:07:45,303 - lava_callback - INFO - Starting async job submit for node 69c1e38f0efaf21d93106014 2026-03-24 01:07:45,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:07:45,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e38f0efaf21d93106014 HTTP/1.1" 200 None 2026-03-24 01:07:45,346 - lava_callback - DEBUG - Processing node 69c1e38f0efaf21d93106014: baseline-arm64 2026-03-24 01:07:45,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:07:45,749 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:07:45,751 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1e38f0efaf21d93106014/log.txt.gz 2026-03-24 01:07:45,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:07:46,144 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:07:46,146 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1e38f0efaf21d93106014/lava-logs.yaml 2026-03-24 01:07:46,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:07:46,288 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:07:46,290 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1e38f0efaf21d93106014/lava_callback.json.gz 2026-03-24 01:07:46,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:07:46,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e38f0efaf21d93106014 HTTP/1.1" 200 None 2026-03-24 01:07:46,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:07:46,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-24 01:07:46,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:07:46,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e38f0efaf21d93106014 HTTP/1.1" 200 None 2026-03-24 01:07:46,373 - lava_callback - INFO - Completed processing callback for node 69c1e38f0efaf21d93106014 INFO: 172.21.0.1:54656 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:07:47,335 - lava_callback - INFO - Received callback for node 69c1e3870efaf21d93105ffd INFO: 172.21.0.1:54666 - "POST /node/69c1e3870efaf21d93105ffd HTTP/1.0" 202 Accepted 2026-03-24 01:07:47,341 - lava_callback - INFO - Starting async job submit for node 69c1e3870efaf21d93105ffd 2026-03-24 01:07:47,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:07:47,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3870efaf21d93105ffd HTTP/1.1" 200 None 2026-03-24 01:07:47,378 - lava_callback - DEBUG - Processing node 69c1e3870efaf21d93105ffd: baseline-arm64 2026-03-24 01:07:47,399 - lava_callback - INFO - Received callback for node 69c1e3bb0efaf21d93106099 INFO: 172.21.0.1:54672 - "POST /node/69c1e3bb0efaf21d93106099 HTTP/1.0" 202 Accepted 2026-03-24 01:07:47,426 - lava_callback - INFO - Starting async job submit for node 69c1e3bb0efaf21d93106099 2026-03-24 01:07:47,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:07:47,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3bb0efaf21d93106099 HTTP/1.1" 200 None 2026-03-24 01:07:47,567 - lava_callback - DEBUG - Processing node 69c1e3bb0efaf21d93106099: baseline-arm64 2026-03-24 01:07:47,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:07:47,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:07:47,993 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:07:47,995 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:07:47,997 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1e3870efaf21d93105ffd/log.txt.gz 2026-03-24 01:07:47,997 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1e3bb0efaf21d93106099/log.txt.gz 2026-03-24 01:07:48,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:07:48,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:07:48,424 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:07:48,426 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1e3bb0efaf21d93106099/lava-logs.yaml 2026-03-24 01:07:48,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:07:48,482 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:07:48,485 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1e3870efaf21d93105ffd/lava-logs.yaml 2026-03-24 01:07:48,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:07:48,649 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:07:48,651 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1e3bb0efaf21d93106099/lava_callback.json.gz 2026-03-24 01:07:48,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:07:48,673 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:07:48,674 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1e3870efaf21d93105ffd/lava_callback.json.gz 2026-03-24 01:07:48,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:07:48,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3bb0efaf21d93106099 HTTP/1.1" 200 None 2026-03-24 01:07:48,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:07:48,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3870efaf21d93105ffd HTTP/1.1" 200 None 2026-03-24 01:07:48,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-24 01:07:48,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:07:48,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:07:48,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-24 01:07:48,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:07:48,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e3870efaf21d93105ffd HTTP/1.1" 200 None 2026-03-24 01:07:48,770 - lava_callback - INFO - Completed processing callback for node 69c1e3870efaf21d93105ffd 2026-03-24 01:07:48,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e3bb0efaf21d93106099 HTTP/1.1" 200 None 2026-03-24 01:07:48,815 - lava_callback - INFO - Completed processing callback for node 69c1e3bb0efaf21d93106099 2026-03-24 01:07:54,260 - lava_callback - INFO - Received callback for node 69c1e3810efaf21d93105fd9 INFO: 172.21.0.1:53158 - "POST /node/69c1e3810efaf21d93105fd9 HTTP/1.0" 202 Accepted 2026-03-24 01:07:54,266 - lava_callback - INFO - Starting async job submit for node 69c1e3810efaf21d93105fd9 2026-03-24 01:07:54,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:07:54,302 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3810efaf21d93105fd9 HTTP/1.1" 200 None 2026-03-24 01:07:54,303 - lava_callback - DEBUG - Processing node 69c1e3810efaf21d93105fd9: baseline-arm64 2026-03-24 01:07:54,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:07:54,634 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:07:54,636 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1e3810efaf21d93105fd9/log.txt.gz 2026-03-24 01:07:54,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:07:54,933 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:07:54,935 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1e3810efaf21d93105fd9/lava-logs.yaml 2026-03-24 01:07:54,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:07:55,103 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:07:55,104 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1e3810efaf21d93105fd9/lava_callback.json.gz 2026-03-24 01:07:55,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:07:55,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3810efaf21d93105fd9 HTTP/1.1" 200 None 2026-03-24 01:07:55,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:07:55,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-24 01:07:55,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:07:55,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e3810efaf21d93105fd9 HTTP/1.1" 200 None 2026-03-24 01:07:55,209 - lava_callback - INFO - Completed processing callback for node 69c1e3810efaf21d93105fd9 2026-03-24 01:07:55,332 - lava_callback - INFO - Received callback for node 69c1e3820efaf21d93105fdf INFO: 172.21.0.1:53160 - "POST /node/69c1e3820efaf21d93105fdf HTTP/1.0" 202 Accepted 2026-03-24 01:07:55,342 - lava_callback - INFO - Starting async job submit for node 69c1e3820efaf21d93105fdf 2026-03-24 01:07:55,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:07:55,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3820efaf21d93105fdf HTTP/1.1" 200 None 2026-03-24 01:07:55,399 - lava_callback - DEBUG - Processing node 69c1e3820efaf21d93105fdf: baseline-arm64 2026-03-24 01:07:55,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:07:55,821 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:07:55,822 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1e3820efaf21d93105fdf/log.txt.gz 2026-03-24 01:07:55,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:07:56,165 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:07:56,167 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1e3820efaf21d93105fdf/lava-logs.yaml 2026-03-24 01:07:56,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:07:56,297 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:07:56,299 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1e3820efaf21d93105fdf/lava_callback.json.gz 2026-03-24 01:07:56,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:07:56,351 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3820efaf21d93105fdf HTTP/1.1" 200 None 2026-03-24 01:07:56,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:07:56,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-24 01:07:56,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:07:56,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e3820efaf21d93105fdf HTTP/1.1" 200 None 2026-03-24 01:07:56,401 - lava_callback - INFO - Completed processing callback for node 69c1e3820efaf21d93105fdf INFO: 172.21.0.1:53164 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:07:56,622 - lava_callback - INFO - Received callback for node 69c1e3a50efaf21d93106066 INFO: 172.21.0.1:53174 - "POST /node/69c1e3a50efaf21d93106066 HTTP/1.0" 202 Accepted 2026-03-24 01:07:56,629 - lava_callback - INFO - Starting async job submit for node 69c1e3a50efaf21d93106066 2026-03-24 01:07:56,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:07:56,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3a50efaf21d93106066 HTTP/1.1" 200 None 2026-03-24 01:07:56,668 - lava_callback - DEBUG - Processing node 69c1e3a50efaf21d93106066: baseline-nfs-arm64 2026-03-24 01:07:57,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:07:57,179 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:07:57,181 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c1e3a50efaf21d93106066/log.txt.gz 2026-03-24 01:07:57,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:07:57,590 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:07:57,592 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c1e3a50efaf21d93106066/lava-logs.yaml 2026-03-24 01:07:57,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:07:57,745 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:07:57,747 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c1e3a50efaf21d93106066/lava_callback.json.gz 2026-03-24 01:07:57,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:07:57,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3a50efaf21d93106066 HTTP/1.1" 200 None 2026-03-24 01:07:57,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:07:57,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-24 01:07:57,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:07:57,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e3a50efaf21d93106066 HTTP/1.1" 200 None 2026-03-24 01:07:57,836 - lava_callback - INFO - Completed processing callback for node 69c1e3a50efaf21d93106066 2026-03-24 01:07:57,954 - lava_callback - INFO - Received callback for node 69c1e3920efaf21d9310601a INFO: 172.21.0.1:53180 - "POST /node/69c1e3920efaf21d9310601a HTTP/1.0" 202 Accepted 2026-03-24 01:07:57,960 - lava_callback - INFO - Starting async job submit for node 69c1e3920efaf21d9310601a 2026-03-24 01:07:57,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:07:58,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3920efaf21d9310601a HTTP/1.1" 200 None 2026-03-24 01:07:58,007 - lava_callback - DEBUG - Processing node 69c1e3920efaf21d9310601a: baseline-arm64 2026-03-24 01:07:58,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:07:58,483 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:07:58,485 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1e3920efaf21d9310601a/log.txt.gz 2026-03-24 01:07:58,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:07:58,948 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:07:58,950 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1e3920efaf21d9310601a/lava-logs.yaml 2026-03-24 01:07:58,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:07:59,091 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:07:59,093 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1e3920efaf21d9310601a/lava_callback.json.gz 2026-03-24 01:07:59,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:07:59,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3920efaf21d9310601a HTTP/1.1" 200 None 2026-03-24 01:07:59,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:07:59,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-24 01:07:59,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:07:59,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e3920efaf21d9310601a HTTP/1.1" 200 None 2026-03-24 01:07:59,178 - lava_callback - INFO - Completed processing callback for node 69c1e3920efaf21d9310601a 2026-03-24 01:08:00,290 - lava_callback - INFO - Received callback for node 69c1e38b0efaf21d93106006 INFO: 172.21.0.1:49776 - "POST /node/69c1e38b0efaf21d93106006 HTTP/1.0" 202 Accepted 2026-03-24 01:08:00,298 - lava_callback - INFO - Starting async job submit for node 69c1e38b0efaf21d93106006 2026-03-24 01:08:00,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:08:00,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e38b0efaf21d93106006 HTTP/1.1" 200 None 2026-03-24 01:08:00,337 - lava_callback - DEBUG - Processing node 69c1e38b0efaf21d93106006: baseline-arm64 2026-03-24 01:08:00,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:08:00,909 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:08:00,911 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1e38b0efaf21d93106006/log.txt.gz 2026-03-24 01:08:01,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:08:01,386 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:08:01,388 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1e38b0efaf21d93106006/lava-logs.yaml 2026-03-24 01:08:01,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:08:01,578 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:08:01,580 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1e38b0efaf21d93106006/lava_callback.json.gz 2026-03-24 01:08:01,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:08:01,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e38b0efaf21d93106006 HTTP/1.1" 200 None 2026-03-24 01:08:01,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:08:01,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-24 01:08:01,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:08:01,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e38b0efaf21d93106006 HTTP/1.1" 200 None 2026-03-24 01:08:01,766 - lava_callback - INFO - Completed processing callback for node 69c1e38b0efaf21d93106006 2026-03-24 01:08:02,857 - lava_callback - INFO - Received callback for node 69c1e3850efaf21d93105fe6 INFO: 172.21.0.1:49786 - "POST /node/69c1e3850efaf21d93105fe6 HTTP/1.0" 202 Accepted 2026-03-24 01:08:02,864 - lava_callback - INFO - Starting async job submit for node 69c1e3850efaf21d93105fe6 2026-03-24 01:08:02,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:08:02,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3850efaf21d93105fe6 HTTP/1.1" 200 None 2026-03-24 01:08:02,911 - lava_callback - DEBUG - Processing node 69c1e3850efaf21d93105fe6: baseline-arm64 2026-03-24 01:08:03,127 - lava_callback - WARNING - Failed to upload log 2026-03-24 01:08:03,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:08:03,407 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:08:03,409 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1e3850efaf21d93105fe6/lava-logs.yaml 2026-03-24 01:08:03,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:08:03,543 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:08:03,545 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1e3850efaf21d93105fe6/lava_callback.json.gz 2026-03-24 01:08:03,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:08:03,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3850efaf21d93105fe6 HTTP/1.1" 200 None 2026-03-24 01:08:03,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:08:03,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-24 01:08:03,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:08:03,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e3850efaf21d93105fe6 HTTP/1.1" 200 None 2026-03-24 01:08:03,697 - lava_callback - INFO - Completed processing callback for node 69c1e3850efaf21d93105fe6 INFO: 172.21.0.1:49796 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:08:09,100 - lava_callback - INFO - Received callback for node 69c1e39f0efaf21d93106034 INFO: 172.21.0.1:49810 - "POST /node/69c1e39f0efaf21d93106034 HTTP/1.0" 202 Accepted 2026-03-24 01:08:09,107 - lava_callback - INFO - Starting async job submit for node 69c1e39f0efaf21d93106034 2026-03-24 01:08:09,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:08:09,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e39f0efaf21d93106034 HTTP/1.1" 200 None 2026-03-24 01:08:09,152 - lava_callback - DEBUG - Processing node 69c1e39f0efaf21d93106034: baseline-nfs-arm64 INFO: 172.21.0.1:49826 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:08:09,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:08:09,806 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:08:09,808 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c1e39f0efaf21d93106034/log.txt.gz 2026-03-24 01:08:10,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:08:10,384 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:08:10,386 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c1e39f0efaf21d93106034/lava-logs.yaml 2026-03-24 01:08:10,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:08:10,549 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:08:10,551 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c1e39f0efaf21d93106034/lava_callback.json.gz 2026-03-24 01:08:10,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:08:10,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:08:10,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e39f0efaf21d93106034 HTTP/1.1" 200 None 2026-03-24 01:08:10,599 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:08:10,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-24 01:08:10,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-24 01:08:10,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:08:10,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e39f0efaf21d93106034 HTTP/1.1" 200 None 2026-03-24 01:08:10,644 - lava_callback - INFO - Completed processing callback for node 69c1e39f0efaf21d93106034 2026-03-24 01:08:12,067 - lava_callback - INFO - Received callback for node 69c1e3a30efaf21d93106042 INFO: 172.21.0.1:44634 - "POST /node/69c1e3a30efaf21d93106042 HTTP/1.0" 202 Accepted 2026-03-24 01:08:12,073 - lava_callback - INFO - Starting async job submit for node 69c1e3a30efaf21d93106042 2026-03-24 01:08:12,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:08:12,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3a30efaf21d93106042 HTTP/1.1" 200 None 2026-03-24 01:08:12,112 - lava_callback - DEBUG - Processing node 69c1e3a30efaf21d93106042: baseline-nfs-arm64 2026-03-24 01:08:12,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:08:12,594 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:08:12,596 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c1e3a30efaf21d93106042/log.txt.gz 2026-03-24 01:08:12,662 - lava_callback - INFO - Received callback for node 69c1e39d0efaf21d93106031 INFO: 172.21.0.1:44640 - "POST /node/69c1e39d0efaf21d93106031 HTTP/1.0" 202 Accepted 2026-03-24 01:08:12,678 - lava_callback - INFO - Starting async job submit for node 69c1e39d0efaf21d93106031 2026-03-24 01:08:12,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:08:12,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e39d0efaf21d93106031 HTTP/1.1" 200 None 2026-03-24 01:08:12,806 - lava_callback - DEBUG - Processing node 69c1e39d0efaf21d93106031: baseline-nfs-arm64 2026-03-24 01:08:12,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:08:13,232 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:08:13,269 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c1e3a30efaf21d93106042/lava-logs.yaml 2026-03-24 01:08:13,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:08:13,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:08:13,558 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:08:13,560 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c1e3a30efaf21d93106042/lava_callback.json.gz 2026-03-24 01:08:13,589 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:08:13,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:08:13,591 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c1e39d0efaf21d93106031/log.txt.gz 2026-03-24 01:08:13,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3a30efaf21d93106042 HTTP/1.1" 200 None 2026-03-24 01:08:13,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:08:13,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-24 01:08:13,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:08:13,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e3a30efaf21d93106042 HTTP/1.1" 200 None 2026-03-24 01:08:13,861 - lava_callback - INFO - Completed processing callback for node 69c1e3a30efaf21d93106042 2026-03-24 01:08:13,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:08:14,131 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:08:14,133 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c1e39d0efaf21d93106031/lava-logs.yaml 2026-03-24 01:08:14,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:08:14,341 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:08:14,342 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c1e39d0efaf21d93106031/lava_callback.json.gz 2026-03-24 01:08:14,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:08:14,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e39d0efaf21d93106031 HTTP/1.1" 200 None 2026-03-24 01:08:14,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:08:14,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-24 01:08:14,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:08:14,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e39d0efaf21d93106031 HTTP/1.1" 200 None 2026-03-24 01:08:14,426 - lava_callback - INFO - Completed processing callback for node 69c1e39d0efaf21d93106031 INFO: 172.21.0.1:44642 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44654 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:08:20,257 - lava_callback - INFO - Received callback for node 69c1e3c90efaf21d931060e2 INFO: 172.21.0.1:54072 - "POST /node/69c1e3c90efaf21d931060e2 HTTP/1.0" 202 Accepted 2026-03-24 01:08:20,263 - lava_callback - INFO - Starting async job submit for node 69c1e3c90efaf21d931060e2 2026-03-24 01:08:20,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:08:20,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3c90efaf21d931060e2 HTTP/1.1" 200 None 2026-03-24 01:08:20,297 - lava_callback - DEBUG - Processing node 69c1e3c90efaf21d931060e2: baseline-arm64-pengutronix 2026-03-24 01:08:20,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:08:20,616 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:08:20,617 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c1e3c90efaf21d931060e2/log.txt.gz 2026-03-24 01:08:20,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:08:20,905 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:08:20,908 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c1e3c90efaf21d931060e2/lava-logs.yaml 2026-03-24 01:08:20,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:08:21,063 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:08:21,064 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c1e3c90efaf21d931060e2/lava_callback.json.gz 2026-03-24 01:08:21,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:08:21,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3c90efaf21d931060e2 HTTP/1.1" 200 None 2026-03-24 01:08:21,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:08:21,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-24 01:08:21,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:08:21,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e3c90efaf21d931060e2 HTTP/1.1" 200 None 2026-03-24 01:08:21,151 - lava_callback - INFO - Completed processing callback for node 69c1e3c90efaf21d931060e2 2026-03-24 01:08:22,071 - lava_callback - INFO - Received callback for node 69c1e3a60efaf21d93106069 INFO: 172.21.0.1:54076 - "POST /node/69c1e3a60efaf21d93106069 HTTP/1.0" 202 Accepted 2026-03-24 01:08:22,077 - lava_callback - INFO - Starting async job submit for node 69c1e3a60efaf21d93106069 2026-03-24 01:08:22,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:08:22,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3a60efaf21d93106069 HTTP/1.1" 200 None 2026-03-24 01:08:22,116 - lava_callback - DEBUG - Processing node 69c1e3a60efaf21d93106069: baseline-nfs-arm64 2026-03-24 01:08:22,296 - lava_callback - INFO - Received callback for node 69c1e3a10efaf21d9310603c INFO: 172.21.0.1:54092 - "POST /node/69c1e3a10efaf21d9310603c HTTP/1.0" 202 Accepted 2026-03-24 01:08:22,334 - lava_callback - INFO - Starting async job submit for node 69c1e3a10efaf21d9310603c 2026-03-24 01:08:22,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:08:22,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3a10efaf21d9310603c HTTP/1.1" 200 None 2026-03-24 01:08:22,484 - lava_callback - DEBUG - Processing node 69c1e3a10efaf21d9310603c: baseline-nfs-arm64 2026-03-24 01:08:22,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:08:22,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:08:22,979 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:08:22,981 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c1e3a60efaf21d93106069/log.txt.gz 2026-03-24 01:08:23,094 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:08:23,143 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c1e3a10efaf21d9310603c/log.txt.gz 2026-03-24 01:08:23,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:08:23,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:08:23,663 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:08:23,666 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c1e3a60efaf21d93106069/lava-logs.yaml 2026-03-24 01:08:23,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:08:23,747 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:08:23,749 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c1e3a10efaf21d9310603c/lava-logs.yaml 2026-03-24 01:08:23,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:08:23,844 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:08:23,846 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c1e3a60efaf21d93106069/lava_callback.json.gz 2026-03-24 01:08:23,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:08:23,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3a60efaf21d93106069 HTTP/1.1" 200 None 2026-03-24 01:08:23,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:08:23,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-24 01:08:23,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:08:23,911 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:08:23,913 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c1e3a10efaf21d9310603c/lava_callback.json.gz 2026-03-24 01:08:23,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:08:23,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3a10efaf21d9310603c HTTP/1.1" 200 None 2026-03-24 01:08:23,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:08:23,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e3a60efaf21d93106069 HTTP/1.1" 200 None 2026-03-24 01:08:23,976 - lava_callback - INFO - Completed processing callback for node 69c1e3a60efaf21d93106069 2026-03-24 01:08:24,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-24 01:08:24,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:08:24,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e3a10efaf21d9310603c HTTP/1.1" 200 None 2026-03-24 01:08:24,073 - lava_callback - INFO - Completed processing callback for node 69c1e3a10efaf21d9310603c INFO: 172.21.0.1:54108 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:08:27,800 - lava_callback - INFO - Received callback for node 69c1e3ca0efaf21d931060e5 INFO: 172.21.0.1:54112 - "POST /node/69c1e3ca0efaf21d931060e5 HTTP/1.0" 202 Accepted 2026-03-24 01:08:27,806 - lava_callback - INFO - Starting async job submit for node 69c1e3ca0efaf21d931060e5 2026-03-24 01:08:27,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:08:27,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3ca0efaf21d931060e5 HTTP/1.1" 200 None 2026-03-24 01:08:27,841 - lava_callback - DEBUG - Processing node 69c1e3ca0efaf21d931060e5: baseline-arm64-pengutronix 2026-03-24 01:08:28,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:08:28,165 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:08:28,166 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c1e3ca0efaf21d931060e5/log.txt.gz 2026-03-24 01:08:28,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:08:28,472 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:08:28,474 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c1e3ca0efaf21d931060e5/lava-logs.yaml 2026-03-24 01:08:28,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:08:28,609 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:08:28,610 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c1e3ca0efaf21d931060e5/lava_callback.json.gz 2026-03-24 01:08:28,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:08:28,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3ca0efaf21d931060e5 HTTP/1.1" 200 None 2026-03-24 01:08:28,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:08:28,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-24 01:08:28,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:08:28,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e3ca0efaf21d931060e5 HTTP/1.1" 200 None 2026-03-24 01:08:28,685 - lava_callback - INFO - Completed processing callback for node 69c1e3ca0efaf21d931060e5 2026-03-24 01:08:31,893 - lava_callback - INFO - Received callback for node 69c1e3d00efaf21d931060eb INFO: 172.21.0.1:60020 - "POST /node/69c1e3d00efaf21d931060eb HTTP/1.0" 202 Accepted 2026-03-24 01:08:31,900 - lava_callback - INFO - Starting async job submit for node 69c1e3d00efaf21d931060eb 2026-03-24 01:08:31,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:08:31,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3d00efaf21d931060eb HTTP/1.1" 200 None 2026-03-24 01:08:31,946 - lava_callback - DEBUG - Processing node 69c1e3d00efaf21d931060eb: baseline-arm64-foundriesio 2026-03-24 01:08:32,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:08:32,323 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:08:32,325 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c1e3d00efaf21d931060eb/log.txt.gz 2026-03-24 01:08:32,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:08:32,705 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:08:32,707 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c1e3d00efaf21d931060eb/lava-logs.yaml 2026-03-24 01:08:32,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:08:32,848 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:08:32,849 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c1e3d00efaf21d931060eb/lava_callback.json.gz 2026-03-24 01:08:32,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:08:32,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3d00efaf21d931060eb HTTP/1.1" 200 None 2026-03-24 01:08:32,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:08:32,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-24 01:08:32,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:08:32,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e3d00efaf21d931060eb HTTP/1.1" 200 None 2026-03-24 01:08:32,953 - lava_callback - INFO - Completed processing callback for node 69c1e3d00efaf21d931060eb INFO: 172.21.0.1:60022 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:08:37,568 - lava_callback - INFO - Received callback for node 69c1e3cb0efaf21d931060e8 INFO: 172.21.0.1:60032 - "POST /node/69c1e3cb0efaf21d931060e8 HTTP/1.0" 202 Accepted 2026-03-24 01:08:37,574 - lava_callback - INFO - Starting async job submit for node 69c1e3cb0efaf21d931060e8 2026-03-24 01:08:37,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:08:37,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3cb0efaf21d931060e8 HTTP/1.1" 200 None 2026-03-24 01:08:37,609 - lava_callback - DEBUG - Processing node 69c1e3cb0efaf21d931060e8: baseline-arm64-pengutronix 2026-03-24 01:08:37,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:08:37,955 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:08:37,957 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c1e3cb0efaf21d931060e8/log.txt.gz 2026-03-24 01:08:38,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:08:38,255 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:08:38,258 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c1e3cb0efaf21d931060e8/lava-logs.yaml 2026-03-24 01:08:38,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:08:38,428 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:08:38,430 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c1e3cb0efaf21d931060e8/lava_callback.json.gz 2026-03-24 01:08:38,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:08:38,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3cb0efaf21d931060e8 HTTP/1.1" 200 None 2026-03-24 01:08:38,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:08:38,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-24 01:08:38,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:08:38,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e3cb0efaf21d931060e8 HTTP/1.1" 200 None 2026-03-24 01:08:38,574 - lava_callback - INFO - Completed processing callback for node 69c1e3cb0efaf21d931060e8 2026-03-24 01:08:40,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:08:40,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-24 01:08:41,679 - lava_callback - INFO - Received callback for node 69c1e3940efaf21d93106022 INFO: 172.21.0.1:57334 - "POST /node/69c1e3940efaf21d93106022 HTTP/1.0" 202 Accepted 2026-03-24 01:08:41,685 - lava_callback - INFO - Starting async job submit for node 69c1e3940efaf21d93106022 2026-03-24 01:08:41,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:08:41,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3940efaf21d93106022 HTTP/1.1" 200 None 2026-03-24 01:08:41,722 - lava_callback - DEBUG - Processing node 69c1e3940efaf21d93106022: baseline-nfs-arm64 2026-03-24 01:08:42,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:08:42,232 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:08:42,234 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c1e3940efaf21d93106022/log.txt.gz 2026-03-24 01:08:42,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:08:42,879 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:08:42,881 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c1e3940efaf21d93106022/lava-logs.yaml 2026-03-24 01:08:42,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:08:43,094 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:08:43,096 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c1e3940efaf21d93106022/lava_callback.json.gz 2026-03-24 01:08:43,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:08:43,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3940efaf21d93106022 HTTP/1.1" 200 None 2026-03-24 01:08:43,140 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:08:43,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-24 01:08:43,163 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:08:43,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e3940efaf21d93106022 HTTP/1.1" 200 None 2026-03-24 01:08:43,194 - lava_callback - INFO - Completed processing callback for node 69c1e3940efaf21d93106022 INFO: 172.21.0.1:57350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55960 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:09:00,576 - lava_callback - INFO - Received callback for node 69c1e3d10efaf21d931060ee INFO: 172.21.0.1:33166 - "POST /node/69c1e3d10efaf21d931060ee HTTP/1.0" 202 Accepted 2026-03-24 01:09:00,583 - lava_callback - INFO - Starting async job submit for node 69c1e3d10efaf21d931060ee 2026-03-24 01:09:00,611 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:09:00,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3d10efaf21d931060ee HTTP/1.1" 200 None 2026-03-24 01:09:00,634 - lava_callback - DEBUG - Processing node 69c1e3d10efaf21d931060ee: baseline-arm64-foundriesio 2026-03-24 01:09:01,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:09:01,239 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:09:01,241 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c1e3d10efaf21d931060ee/log.txt.gz 2026-03-24 01:09:01,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:09:01,505 - lava_callback - INFO - Received callback for node 69c1dcd90efaf21d931036a0 INFO: 172.21.0.1:33168 - "POST /node/69c1dcd90efaf21d931036a0 HTTP/1.0" 202 Accepted 2026-03-24 01:09:01,560 - lava_callback - INFO - Starting async job submit for node 69c1dcd90efaf21d931036a0 2026-03-24 01:09:01,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:09:01,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dcd90efaf21d931036a0 HTTP/1.1" 200 None 2026-03-24 01:09:01,611 - lava_callback - DEBUG - Processing node 69c1dcd90efaf21d931036a0: ltp-smoke 2026-03-24 01:09:01,702 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:09:01,717 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c1e3d10efaf21d931060ee/lava-logs.yaml 2026-03-24 01:09:01,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:09:01,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:09:01,879 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:09:01,880 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c1dcd90efaf21d931036a0/log.txt.gz 2026-03-24 01:09:01,922 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:09:01,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:09:01,937 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c1e3d10efaf21d931060ee/lava_callback.json.gz 2026-03-24 01:09:01,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:09:01,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3d10efaf21d931060ee HTTP/1.1" 200 None 2026-03-24 01:09:01,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:09:02,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-24 01:09:02,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:09:02,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e3d10efaf21d931060ee HTTP/1.1" 200 None 2026-03-24 01:09:02,057 - lava_callback - INFO - Completed processing callback for node 69c1e3d10efaf21d931060ee 2026-03-24 01:09:02,168 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:09:02,170 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c1dcd90efaf21d931036a0/lava-logs.yaml 2026-03-24 01:09:02,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:09:02,316 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:09:02,318 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c1dcd90efaf21d931036a0/lava_callback.json.gz 2026-03-24 01:09:02,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:09:02,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dcd90efaf21d931036a0 HTTP/1.1" 200 None 2026-03-24 01:09:02,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:09:02,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-24 01:09:02,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:09:02,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1dcd90efaf21d931036a0 HTTP/1.1" 200 None 2026-03-24 01:09:02,467 - lava_callback - INFO - Completed processing callback for node 69c1dcd90efaf21d931036a0 2026-03-24 01:09:02,771 - lava_callback - INFO - Received callback for node 69c1e3c50efaf21d931060d7 INFO: 172.21.0.1:33178 - "POST /node/69c1e3c50efaf21d931060d7 HTTP/1.0" 202 Accepted 2026-03-24 01:09:02,778 - lava_callback - INFO - Starting async job submit for node 69c1e3c50efaf21d931060d7 2026-03-24 01:09:02,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:09:02,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3c50efaf21d931060d7 HTTP/1.1" 200 None 2026-03-24 01:09:02,815 - lava_callback - DEBUG - Processing node 69c1e3c50efaf21d931060d7: baseline-arm64-clabbe 2026-03-24 01:09:03,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:09:03,176 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:09:03,177 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69c1e3c50efaf21d931060d7/log.txt.gz 2026-03-24 01:09:03,334 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:09:03,525 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:09:03,527 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69c1e3c50efaf21d931060d7/lava-logs.yaml 2026-03-24 01:09:03,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:09:03,664 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:09:03,665 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69c1e3c50efaf21d931060d7/lava_callback.json.gz 2026-03-24 01:09:03,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:09:03,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3c50efaf21d931060d7 HTTP/1.1" 200 None 2026-03-24 01:09:03,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:09:03,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-24 01:09:03,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:09:03,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e3c50efaf21d931060d7 HTTP/1.1" 200 None 2026-03-24 01:09:03,759 - lava_callback - INFO - Completed processing callback for node 69c1e3c50efaf21d931060d7 INFO: 172.21.0.1:33192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33206 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:09:10,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:09:10,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.21.0.1:53314 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42930 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:09:40,385 - lava_callback - INFO - Received callback for node 69c1e43f0efaf21d93106212 INFO: 172.21.0.1:38766 - "POST /node/69c1e43f0efaf21d93106212 HTTP/1.0" 202 Accepted 2026-03-24 01:09:40,391 - lava_callback - INFO - Starting async job submit for node 69c1e43f0efaf21d93106212 2026-03-24 01:09:40,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:09:40,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e43f0efaf21d93106212 HTTP/1.1" 200 None 2026-03-24 01:09:40,408 - lava_callback - DEBUG - Processing node 69c1e43f0efaf21d93106212: baseline-nfs-arm64 2026-03-24 01:09:40,409 - lava_callback - WARNING - Failed to upload log 2026-03-24 01:09:40,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:09:40,554 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:09:40,556 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c1e43f0efaf21d93106212/lava-logs.yaml 2026-03-24 01:09:40,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:09:40,692 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:09:40,694 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c1e43f0efaf21d93106212/lava_callback.json.gz 2026-03-24 01:09:40,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:09:40,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e43f0efaf21d93106212 HTTP/1.1" 200 None 2026-03-24 01:09:40,711 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:09:40,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-24 01:09:40,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:09:40,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e43f0efaf21d93106212 HTTP/1.1" 200 None 2026-03-24 01:09:40,756 - lava_callback - INFO - Completed processing callback for node 69c1e43f0efaf21d93106212 INFO: 172.21.0.1:38776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52006 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:10:07,860 - lava_callback - INFO - Received callback for node 69c1e3860efaf21d93105ffa INFO: 172.21.0.1:52016 - "POST /node/69c1e3860efaf21d93105ffa HTTP/1.0" 202 Accepted 2026-03-24 01:10:07,866 - lava_callback - INFO - Starting async job submit for node 69c1e3860efaf21d93105ffa 2026-03-24 01:10:07,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:10:07,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3860efaf21d93105ffa HTTP/1.1" 200 None 2026-03-24 01:10:07,899 - lava_callback - DEBUG - Processing node 69c1e3860efaf21d93105ffa: baseline-arm64 2026-03-24 01:10:08,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:10:08,414 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:10:08,417 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1e3860efaf21d93105ffa/log.txt.gz 2026-03-24 01:10:08,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:10:08,856 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:10:08,858 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1e3860efaf21d93105ffa/lava-logs.yaml 2026-03-24 01:10:08,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:10:09,016 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:10:09,018 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1e3860efaf21d93105ffa/lava_callback.json.gz 2026-03-24 01:10:09,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:10:09,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3860efaf21d93105ffa HTTP/1.1" 200 None 2026-03-24 01:10:09,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:10:09,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-24 01:10:09,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:10:09,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e3860efaf21d93105ffa HTTP/1.1" 200 None 2026-03-24 01:10:09,104 - lava_callback - INFO - Completed processing callback for node 69c1e3860efaf21d93105ffa INFO: 172.21.0.1:52018 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:10:10,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:10:10,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-24 01:10:14,569 - lava_callback - INFO - Received callback for node 69bf4173f1004e82292e3906 INFO: 172.21.0.1:39756 - "POST /node/69bf4173f1004e82292e3906 HTTP/1.0" 202 Accepted 2026-03-24 01:10:14,576 - lava_callback - INFO - Starting async job submit for node 69bf4173f1004e82292e3906 2026-03-24 01:10:14,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:10:14,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf4173f1004e82292e3906 HTTP/1.1" 200 None 2026-03-24 01:10:14,592 - lava_callback - DEBUG - Processing node 69bf4173f1004e82292e3906: baseline-arm64-baylibre 2026-03-24 01:10:14,592 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf4173f1004e82292e3906 2026-03-24 01:10:14,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:10:14,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf4173f1004e82292e3906 HTTP/1.1" 200 None 2026-03-24 01:10:14,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:10:14,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-24 01:10:14,625 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:10:14,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf4173f1004e82292e3906 HTTP/1.1" 200 None 2026-03-24 01:10:14,651 - lava_callback - ERROR - Error processing callback for node 69bf4173f1004e82292e3906: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39758 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59722 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:10:26,692 - lava_callback - INFO - Received callback for node 69c1e45f0efaf21d9310622f INFO: 172.21.0.1:59736 - "POST /node/69c1e45f0efaf21d9310622f HTTP/1.0" 202 Accepted 2026-03-24 01:10:26,698 - lava_callback - INFO - Starting async job submit for node 69c1e45f0efaf21d9310622f 2026-03-24 01:10:26,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:10:26,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e45f0efaf21d9310622f HTTP/1.1" 200 None 2026-03-24 01:10:26,731 - lava_callback - DEBUG - Processing node 69c1e45f0efaf21d9310622f: baseline-x86 2026-03-24 01:10:26,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:10:27,029 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:10:27,031 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c1e45f0efaf21d9310622f/log.txt.gz 2026-03-24 01:10:27,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:10:27,348 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:10:27,350 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c1e45f0efaf21d9310622f/lava-logs.yaml 2026-03-24 01:10:27,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:10:27,500 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:10:27,502 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c1e45f0efaf21d9310622f/lava_callback.json.gz 2026-03-24 01:10:27,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:10:27,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e45f0efaf21d9310622f HTTP/1.1" 200 None 2026-03-24 01:10:27,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:10:27,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d85a0efaf21d9310356c HTTP/1.1" 200 None 2026-03-24 01:10:27,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:10:27,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e45f0efaf21d9310622f HTTP/1.1" 200 None 2026-03-24 01:10:27,604 - lava_callback - INFO - Completed processing callback for node 69c1e45f0efaf21d9310622f 2026-03-24 01:10:27,709 - lava_callback - INFO - Received callback for node 69c1e45d0efaf21d93106226 INFO: 172.21.0.1:59750 - "POST /node/69c1e45d0efaf21d93106226 HTTP/1.0" 202 Accepted 2026-03-24 01:10:27,719 - lava_callback - INFO - Starting async job submit for node 69c1e45d0efaf21d93106226 2026-03-24 01:10:27,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:10:27,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e45d0efaf21d93106226 HTTP/1.1" 200 None 2026-03-24 01:10:27,776 - lava_callback - DEBUG - Processing node 69c1e45d0efaf21d93106226: baseline-x86 2026-03-24 01:10:28,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:10:28,161 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:10:28,162 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c1e45d0efaf21d93106226/log.txt.gz 2026-03-24 01:10:28,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:10:28,440 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:10:28,442 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c1e45d0efaf21d93106226/lava-logs.yaml 2026-03-24 01:10:28,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:10:28,583 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:10:28,585 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c1e45d0efaf21d93106226/lava_callback.json.gz 2026-03-24 01:10:28,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:10:28,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e45d0efaf21d93106226 HTTP/1.1" 200 None 2026-03-24 01:10:28,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:10:28,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d85a0efaf21d9310356c HTTP/1.1" 200 None 2026-03-24 01:10:28,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:10:28,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e45d0efaf21d93106226 HTTP/1.1" 200 None 2026-03-24 01:10:28,671 - lava_callback - INFO - Completed processing callback for node 69c1e45d0efaf21d93106226 2026-03-24 01:10:34,540 - lava_callback - INFO - Received callback for node 69bf4178f1004e82292e3912 INFO: 172.21.0.1:43152 - "POST /node/69bf4178f1004e82292e3912 HTTP/1.0" 202 Accepted 2026-03-24 01:10:34,546 - lava_callback - INFO - Starting async job submit for node 69bf4178f1004e82292e3912 2026-03-24 01:10:34,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:10:34,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf4178f1004e82292e3912 HTTP/1.1" 200 None 2026-03-24 01:10:34,562 - lava_callback - DEBUG - Processing node 69bf4178f1004e82292e3912: baseline-arm64-baylibre 2026-03-24 01:10:34,562 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf4178f1004e82292e3912 2026-03-24 01:10:34,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:10:34,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf4178f1004e82292e3912 HTTP/1.1" 200 None 2026-03-24 01:10:34,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:10:34,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-24 01:10:34,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:10:34,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf4178f1004e82292e3912 HTTP/1.1" 200 None 2026-03-24 01:10:34,628 - lava_callback - ERROR - Error processing callback for node 69bf4178f1004e82292e3912: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-24 01:10:35,225 - lava_callback - INFO - Received callback for node 69bf4176f1004e82292e390d INFO: 172.21.0.1:43166 - "POST /node/69bf4176f1004e82292e390d HTTP/1.0" 202 Accepted 2026-03-24 01:10:35,231 - lava_callback - INFO - Starting async job submit for node 69bf4176f1004e82292e390d 2026-03-24 01:10:35,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:10:35,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf4176f1004e82292e390d HTTP/1.1" 200 None 2026-03-24 01:10:35,247 - lava_callback - DEBUG - Processing node 69bf4176f1004e82292e390d: baseline-arm64-baylibre 2026-03-24 01:10:35,247 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf4176f1004e82292e390d 2026-03-24 01:10:35,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:10:35,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf4176f1004e82292e390d HTTP/1.1" 200 None 2026-03-24 01:10:35,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:10:35,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-24 01:10:35,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:10:35,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf4176f1004e82292e390d HTTP/1.1" 200 None 2026-03-24 01:10:35,308 - lava_callback - ERROR - Error processing callback for node 69bf4176f1004e82292e390d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-24 01:10:36,324 - lava_callback - INFO - Received callback for node 69bf4188f1004e82292e3958 INFO: 172.21.0.1:43168 - "POST /node/69bf4188f1004e82292e3958 HTTP/1.0" 202 Accepted 2026-03-24 01:10:36,330 - lava_callback - INFO - Starting async job submit for node 69bf4188f1004e82292e3958 2026-03-24 01:10:36,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:10:36,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf4188f1004e82292e3958 HTTP/1.1" 200 None 2026-03-24 01:10:36,345 - lava_callback - DEBUG - Processing node 69bf4188f1004e82292e3958: baseline-arm64-clabbe 2026-03-24 01:10:36,346 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf4188f1004e82292e3958 2026-03-24 01:10:36,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:10:36,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf4188f1004e82292e3958 HTTP/1.1" 200 None 2026-03-24 01:10:36,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:10:36,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-24 01:10:36,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:10:36,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf4188f1004e82292e3958 HTTP/1.1" 200 None 2026-03-24 01:10:36,404 - lava_callback - ERROR - Error processing callback for node 69bf4188f1004e82292e3958: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43174 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:10:36,997 - lava_callback - INFO - Received callback for node 69bf418af1004e82292e395b INFO: 172.21.0.1:43178 - "POST /node/69bf418af1004e82292e395b HTTP/1.0" 202 Accepted 2026-03-24 01:10:37,006 - lava_callback - INFO - Starting async job submit for node 69bf418af1004e82292e395b 2026-03-24 01:10:37,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:10:37,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf418af1004e82292e395b HTTP/1.1" 200 None 2026-03-24 01:10:37,023 - lava_callback - DEBUG - Processing node 69bf418af1004e82292e395b: baseline-arm64-clabbe 2026-03-24 01:10:37,023 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf418af1004e82292e395b 2026-03-24 01:10:37,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:10:37,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf418af1004e82292e395b HTTP/1.1" 200 None 2026-03-24 01:10:37,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:10:37,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-24 01:10:37,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:10:37,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf418af1004e82292e395b HTTP/1.1" 200 None 2026-03-24 01:10:37,075 - lava_callback - ERROR - Error processing callback for node 69bf418af1004e82292e395b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-24 01:10:40,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:10:40,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-24 01:10:41,944 - lava_callback - INFO - Received callback for node 69c1e3960efaf21d93106025 INFO: 172.21.0.1:43074 - "POST /node/69c1e3960efaf21d93106025 HTTP/1.0" 202 Accepted 2026-03-24 01:10:41,950 - lava_callback - INFO - Starting async job submit for node 69c1e3960efaf21d93106025 2026-03-24 01:10:41,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:10:41,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3960efaf21d93106025 HTTP/1.1" 200 None 2026-03-24 01:10:41,991 - lava_callback - DEBUG - Processing node 69c1e3960efaf21d93106025: baseline-nfs-arm64 2026-03-24 01:10:42,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:10:42,549 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:10:42,551 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c1e3960efaf21d93106025/log.txt.gz 2026-03-24 01:10:42,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:10:43,053 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:10:43,055 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c1e3960efaf21d93106025/lava-logs.yaml 2026-03-24 01:10:43,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:10:43,232 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:10:43,234 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c1e3960efaf21d93106025/lava_callback.json.gz 2026-03-24 01:10:43,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:10:43,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3960efaf21d93106025 HTTP/1.1" 200 None 2026-03-24 01:10:43,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:10:43,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-24 01:10:43,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:10:43,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e3960efaf21d93106025 HTTP/1.1" 200 None 2026-03-24 01:10:43,330 - lava_callback - INFO - Completed processing callback for node 69c1e3960efaf21d93106025 INFO: 172.21.0.1:43088 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:10:56,267 - lava_callback - INFO - Received callback for node 69bf418bf1004e82292e395e INFO: 172.21.0.1:50434 - "POST /node/69bf418bf1004e82292e395e HTTP/1.0" 202 Accepted 2026-03-24 01:10:56,272 - lava_callback - INFO - Starting async job submit for node 69bf418bf1004e82292e395e 2026-03-24 01:10:56,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:10:56,287 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf418bf1004e82292e395e HTTP/1.1" 200 None 2026-03-24 01:10:56,288 - lava_callback - DEBUG - Processing node 69bf418bf1004e82292e395e: baseline-arm64-clabbe 2026-03-24 01:10:56,288 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf418bf1004e82292e395e 2026-03-24 01:10:56,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:10:56,302 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf418bf1004e82292e395e HTTP/1.1" 200 None 2026-03-24 01:10:56,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:10:56,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-24 01:10:56,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:10:56,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf418bf1004e82292e395e HTTP/1.1" 200 None 2026-03-24 01:10:56,344 - lava_callback - ERROR - Error processing callback for node 69bf418bf1004e82292e395e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50446 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:10:56,930 - lava_callback - INFO - Received callback for node 69bf418cf1004e82292e3963 INFO: 172.21.0.1:50458 - "POST /node/69bf418cf1004e82292e3963 HTTP/1.0" 202 Accepted 2026-03-24 01:10:56,936 - lava_callback - INFO - Starting async job submit for node 69bf418cf1004e82292e3963 2026-03-24 01:10:56,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:10:56,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf418cf1004e82292e3963 HTTP/1.1" 200 None 2026-03-24 01:10:56,952 - lava_callback - DEBUG - Processing node 69bf418cf1004e82292e3963: baseline-arm64-clabbe 2026-03-24 01:10:56,952 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf418cf1004e82292e3963 2026-03-24 01:10:56,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:10:56,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf418cf1004e82292e3963 HTTP/1.1" 200 None 2026-03-24 01:10:56,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:10:56,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-24 01:10:56,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:10:57,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf418cf1004e82292e3963 HTTP/1.1" 200 None 2026-03-24 01:10:57,008 - lava_callback - ERROR - Error processing callback for node 69bf418cf1004e82292e3963: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-24 01:10:57,598 - lava_callback - INFO - Received callback for node 69bf418ef1004e82292e3969 INFO: 172.21.0.1:50460 - "POST /node/69bf418ef1004e82292e3969 HTTP/1.0" 202 Accepted 2026-03-24 01:10:57,603 - lava_callback - INFO - Starting async job submit for node 69bf418ef1004e82292e3969 2026-03-24 01:10:57,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:10:57,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf418ef1004e82292e3969 HTTP/1.1" 200 None 2026-03-24 01:10:57,619 - lava_callback - DEBUG - Processing node 69bf418ef1004e82292e3969: baseline-arm64-clabbe 2026-03-24 01:10:57,619 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf418ef1004e82292e3969 2026-03-24 01:10:57,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:10:57,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf418ef1004e82292e3969 HTTP/1.1" 200 None 2026-03-24 01:10:57,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:10:57,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-24 01:10:57,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:10:57,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf418ef1004e82292e3969 HTTP/1.1" 200 None 2026-03-24 01:10:57,684 - lava_callback - ERROR - Error processing callback for node 69bf418ef1004e82292e3969: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-24 01:10:58,274 - lava_callback - INFO - Received callback for node 69bf418ef1004e82292e396c INFO: 172.21.0.1:50462 - "POST /node/69bf418ef1004e82292e396c HTTP/1.0" 202 Accepted 2026-03-24 01:10:58,280 - lava_callback - INFO - Starting async job submit for node 69bf418ef1004e82292e396c 2026-03-24 01:10:58,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:10:58,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf418ef1004e82292e396c HTTP/1.1" 200 None 2026-03-24 01:10:58,296 - lava_callback - DEBUG - Processing node 69bf418ef1004e82292e396c: baseline-arm64-clabbe 2026-03-24 01:10:58,296 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf418ef1004e82292e396c 2026-03-24 01:10:58,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:10:58,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf418ef1004e82292e396c HTTP/1.1" 200 None 2026-03-24 01:10:58,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:10:58,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-24 01:10:58,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:10:58,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf418ef1004e82292e396c HTTP/1.1" 200 None 2026-03-24 01:10:58,364 - lava_callback - ERROR - Error processing callback for node 69bf418ef1004e82292e396c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39200 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:11:06,711 - lava_callback - INFO - Received callback for node 69c1e45d0efaf21d93106229 INFO: 172.21.0.1:39206 - "POST /node/69c1e45d0efaf21d93106229 HTTP/1.0" 202 Accepted 2026-03-24 01:11:06,718 - lava_callback - INFO - Starting async job submit for node 69c1e45d0efaf21d93106229 2026-03-24 01:11:06,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:11:06,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e45d0efaf21d93106229 HTTP/1.1" 200 None 2026-03-24 01:11:06,751 - lava_callback - DEBUG - Processing node 69c1e45d0efaf21d93106229: baseline-x86 2026-03-24 01:11:07,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:11:07,160 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:11:07,161 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c1e45d0efaf21d93106229/log.txt.gz 2026-03-24 01:11:07,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:11:07,510 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:11:07,512 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c1e45d0efaf21d93106229/lava-logs.yaml 2026-03-24 01:11:07,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:11:07,640 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:11:07,642 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c1e45d0efaf21d93106229/lava_callback.json.gz 2026-03-24 01:11:07,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:11:07,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e45d0efaf21d93106229 HTTP/1.1" 200 None 2026-03-24 01:11:07,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:11:07,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d85a0efaf21d9310356c HTTP/1.1" 200 None 2026-03-24 01:11:07,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:11:07,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e45d0efaf21d93106229 HTTP/1.1" 200 None 2026-03-24 01:11:07,723 - lava_callback - INFO - Completed processing callback for node 69c1e45d0efaf21d93106229 INFO: 172.21.0.1:39214 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:11:10,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:11:10,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:51584 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51596 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:11:22,084 - lava_callback - INFO - Received callback for node 69c1e45e0efaf21d9310622c INFO: 172.21.0.1:56484 - "POST /node/69c1e45e0efaf21d9310622c HTTP/1.0" 202 Accepted 2026-03-24 01:11:22,090 - lava_callback - INFO - Starting async job submit for node 69c1e45e0efaf21d9310622c 2026-03-24 01:11:22,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:11:22,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e45e0efaf21d9310622c HTTP/1.1" 200 None 2026-03-24 01:11:22,128 - lava_callback - DEBUG - Processing node 69c1e45e0efaf21d9310622c: baseline-x86 2026-03-24 01:11:22,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:11:22,525 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:11:22,527 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c1e45e0efaf21d9310622c/log.txt.gz 2026-03-24 01:11:22,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:11:22,886 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:11:22,888 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c1e45e0efaf21d9310622c/lava-logs.yaml 2026-03-24 01:11:22,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:11:23,037 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:11:23,039 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c1e45e0efaf21d9310622c/lava_callback.json.gz 2026-03-24 01:11:23,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:11:23,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e45e0efaf21d9310622c HTTP/1.1" 200 None 2026-03-24 01:11:23,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:11:23,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d85a0efaf21d9310356c HTTP/1.1" 200 None 2026-03-24 01:11:23,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:11:23,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e45e0efaf21d9310622c HTTP/1.1" 200 None 2026-03-24 01:11:23,115 - lava_callback - INFO - Completed processing callback for node 69c1e45e0efaf21d9310622c INFO: 172.21.0.1:56494 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:11:29,339 - lava_callback - INFO - Received callback for node 69c1e4600efaf21d93106232 INFO: 172.21.0.1:56508 - "POST /node/69c1e4600efaf21d93106232 HTTP/1.0" 202 Accepted 2026-03-24 01:11:29,345 - lava_callback - INFO - Starting async job submit for node 69c1e4600efaf21d93106232 2026-03-24 01:11:29,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:11:29,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e4600efaf21d93106232 HTTP/1.1" 200 None 2026-03-24 01:11:29,381 - lava_callback - DEBUG - Processing node 69c1e4600efaf21d93106232: baseline-x86 2026-03-24 01:11:29,659 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:11:29,815 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:11:29,816 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c1e4600efaf21d93106232/log.txt.gz 2026-03-24 01:11:29,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:11:30,209 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:11:30,211 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c1e4600efaf21d93106232/lava-logs.yaml 2026-03-24 01:11:30,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:11:30,368 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:11:30,370 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c1e4600efaf21d93106232/lava_callback.json.gz 2026-03-24 01:11:30,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:11:30,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e4600efaf21d93106232 HTTP/1.1" 200 None 2026-03-24 01:11:30,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:11:30,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d85a0efaf21d9310356c HTTP/1.1" 200 None 2026-03-24 01:11:30,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:11:30,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e4600efaf21d93106232 HTTP/1.1" 200 None 2026-03-24 01:11:30,449 - lava_callback - INFO - Completed processing callback for node 69c1e4600efaf21d93106232 2026-03-24 01:11:31,722 - lava_callback - INFO - Received callback for node 69c1e09c0efaf21d93105ea9 INFO: 172.21.0.1:47278 - "POST /node/69c1e09c0efaf21d93105ea9 HTTP/1.0" 202 Accepted 2026-03-24 01:11:31,729 - lava_callback - INFO - Starting async job submit for node 69c1e09c0efaf21d93105ea9 2026-03-24 01:11:31,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:11:31,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e09c0efaf21d93105ea9 HTTP/1.1" 200 None 2026-03-24 01:11:31,802 - lava_callback - DEBUG - Processing node 69c1e09c0efaf21d93105ea9: ltp-smoke 2026-03-24 01:11:32,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:11:32,397 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:11:32,399 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c1e09c0efaf21d93105ea9/log.txt.gz 2026-03-24 01:11:32,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:11:32,885 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:11:32,887 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c1e09c0efaf21d93105ea9/lava-logs.yaml 2026-03-24 01:11:32,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:11:33,068 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:11:33,070 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c1e09c0efaf21d93105ea9/lava_callback.json.gz 2026-03-24 01:11:33,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:11:33,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e09c0efaf21d93105ea9 HTTP/1.1" 200 None 2026-03-24 01:11:33,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:11:33,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8340efaf21d93103549 HTTP/1.1" 200 None 2026-03-24 01:11:33,239 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:11:33,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e09c0efaf21d93105ea9 HTTP/1.1" 200 None 2026-03-24 01:11:33,378 - lava_callback - INFO - Completed processing callback for node 69c1e09c0efaf21d93105ea9 INFO: 172.21.0.1:47294 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:11:40,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:11:40,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:37322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48896 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38234 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52968 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:13:28,417 - lava_callback - INFO - Received callback for node 69c1e5040efaf21d931062e7 INFO: 172.21.0.1:52972 - "POST /node/69c1e5040efaf21d931062e7 HTTP/1.0" 202 Accepted 2026-03-24 01:13:28,424 - lava_callback - INFO - Starting async job submit for node 69c1e5040efaf21d931062e7 2026-03-24 01:13:28,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:13:28,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e5040efaf21d931062e7 HTTP/1.1" 200 None 2026-03-24 01:13:28,469 - lava_callback - DEBUG - Processing node 69c1e5040efaf21d931062e7: baseline-arm64 2026-03-24 01:13:28,602 - lava_callback - WARNING - Failed to upload log 2026-03-24 01:13:28,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:13:28,867 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:13:28,869 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1e5040efaf21d931062e7/lava-logs.yaml 2026-03-24 01:13:28,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:13:29,017 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:13:29,019 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1e5040efaf21d931062e7/lava_callback.json.gz 2026-03-24 01:13:29,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:13:29,063 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e5040efaf21d931062e7 HTTP/1.1" 200 None 2026-03-24 01:13:29,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:13:29,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-24 01:13:29,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:13:29,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e5040efaf21d931062e7 HTTP/1.1" 200 None 2026-03-24 01:13:29,167 - lava_callback - INFO - Completed processing callback for node 69c1e5040efaf21d931062e7 INFO: 172.21.0.1:33200 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:13:40,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:13:40,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:42832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52264 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:14:07,038 - lava_callback - INFO - Received callback for node 69c1e3930efaf21d9310601d INFO: 172.21.0.1:52270 - "POST /node/69c1e3930efaf21d9310601d HTTP/1.0" 202 Accepted 2026-03-24 01:14:07,045 - lava_callback - INFO - Starting async job submit for node 69c1e3930efaf21d9310601d 2026-03-24 01:14:07,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:14:07,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3930efaf21d9310601d HTTP/1.1" 200 None 2026-03-24 01:14:07,129 - lava_callback - DEBUG - Processing node 69c1e3930efaf21d9310601d: baseline-arm64 2026-03-24 01:14:07,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:14:07,642 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:14:07,643 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1e3930efaf21d9310601d/log.txt.gz 2026-03-24 01:14:07,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:14:08,096 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:14:08,098 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1e3930efaf21d9310601d/lava-logs.yaml 2026-03-24 01:14:08,107 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:14:08,271 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:14:08,273 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1e3930efaf21d9310601d/lava_callback.json.gz DEBUG: Setup failed for 69c1e3930efaf21d9310601d. Transitting job node result: incomplete -> fail 2026-03-24 01:14:08,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:14:08,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3930efaf21d9310601d HTTP/1.1" 200 None 2026-03-24 01:14:08,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:14:08,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-24 01:14:08,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:14:08,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e3930efaf21d9310601d HTTP/1.1" 200 None 2026-03-24 01:14:08,493 - lava_callback - INFO - Completed processing callback for node 69c1e3930efaf21d9310601d INFO: 172.21.0.1:52284 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:14:10,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:14:10,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:57774 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39880 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:14:57,651 - lava_callback - INFO - Received callback for node 69c1e3970efaf21d93106029 INFO: 172.21.0.1:39894 - "POST /node/69c1e3970efaf21d93106029 HTTP/1.0" 202 Accepted 2026-03-24 01:14:57,662 - lava_callback - INFO - Starting async job submit for node 69c1e3970efaf21d93106029 2026-03-24 01:14:57,694 - lava_callback - INFO - Received callback for node 69c1e18c0efaf21d93105f21 INFO: 172.21.0.1:39902 - "POST /node/69c1e18c0efaf21d93105f21 HTTP/1.0" 202 Accepted 2026-03-24 01:14:57,721 - lava_callback - INFO - Starting async job submit for node 69c1e18c0efaf21d93105f21 2026-03-24 01:14:57,743 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:14:57,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3970efaf21d93106029 HTTP/1.1" 200 None 2026-03-24 01:14:57,807 - lava_callback - DEBUG - Processing node 69c1e3970efaf21d93106029: baseline-nfs-arm64 2026-03-24 01:14:57,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:14:57,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e18c0efaf21d93105f21 HTTP/1.1" 200 None 2026-03-24 01:14:57,968 - lava_callback - DEBUG - Processing node 69c1e18c0efaf21d93105f21: baseline-arm 2026-03-24 01:14:58,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:14:59,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:14:59,347 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:14:59,349 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c1e3970efaf21d93106029/log.txt.gz 2026-03-24 01:14:59,521 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:14:59,557 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c1e18c0efaf21d93105f21/log.txt.gz 2026-03-24 01:14:59,743 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:15:00,182 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:15:00,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:15:00,196 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c1e3970efaf21d93106029/lava-logs.yaml 2026-03-24 01:15:00,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:15:00,689 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:15:00,691 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c1e3970efaf21d93106029/lava_callback.json.gz 2026-03-24 01:15:00,726 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:15:00,737 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c1e18c0efaf21d93105f21/lava-logs.yaml 2026-03-24 01:15:00,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:15:00,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:15:00,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3970efaf21d93106029 HTTP/1.1" 200 None 2026-03-24 01:15:00,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:15:00,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-24 01:15:00,798 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:15:00,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e3970efaf21d93106029 HTTP/1.1" 200 None 2026-03-24 01:15:00,836 - lava_callback - INFO - Completed processing callback for node 69c1e3970efaf21d93106029 2026-03-24 01:15:01,320 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:15:01,323 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c1e18c0efaf21d93105f21/lava_callback.json.gz DEBUG: Setup failed for 69c1e18c0efaf21d93105f21. Transitting job node result: incomplete -> fail 2026-03-24 01:15:01,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:15:01,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e18c0efaf21d93105f21 HTTP/1.1" 200 None 2026-03-24 01:15:01,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:15:01,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-24 01:15:01,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:15:01,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e18c0efaf21d93105f21 HTTP/1.1" 200 None 2026-03-24 01:15:01,684 - lava_callback - INFO - Completed processing callback for node 69c1e18c0efaf21d93105f21 INFO: 172.21.0.1:45896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45908 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:15:10,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:15:10,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:39888 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46360 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35690 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:16:34,010 - lava_callback - INFO - Received callback for node 69c1e5620efaf21d93106331 INFO: 172.21.0.1:41202 - "POST /node/69c1e5620efaf21d93106331 HTTP/1.0" 202 Accepted 2026-03-24 01:16:34,017 - lava_callback - INFO - Starting async job submit for node 69c1e5620efaf21d93106331 2026-03-24 01:16:34,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:16:34,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e5620efaf21d93106331 HTTP/1.1" 200 None 2026-03-24 01:16:34,069 - lava_callback - DEBUG - Processing node 69c1e5620efaf21d93106331: baseline-arm64 2026-03-24 01:16:34,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:16:34,745 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:16:34,747 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1e5620efaf21d93106331/log.txt.gz 2026-03-24 01:16:35,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:16:35,300 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:16:35,302 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1e5620efaf21d93106331/lava-logs.yaml 2026-03-24 01:16:35,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:16:35,484 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:16:35,486 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1e5620efaf21d93106331/lava_callback.json.gz 2026-03-24 01:16:35,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:16:35,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e5620efaf21d93106331 HTTP/1.1" 200 None 2026-03-24 01:16:35,548 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:16:35,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-24 01:16:35,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:16:35,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e5620efaf21d93106331 HTTP/1.1" 200 None 2026-03-24 01:16:35,688 - lava_callback - INFO - Completed processing callback for node 69c1e5620efaf21d93106331 INFO: 172.21.0.1:41208 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:16:40,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:16:40,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:40468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37980 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:17:00,143 - lava_callback - INFO - Received callback for node 69c1e38e0efaf21d93106011 INFO: 172.21.0.1:41746 - "POST /node/69c1e38e0efaf21d93106011 HTTP/1.0" 202 Accepted 2026-03-24 01:17:00,149 - lava_callback - INFO - Starting async job submit for node 69c1e38e0efaf21d93106011 2026-03-24 01:17:00,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:17:00,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e38e0efaf21d93106011 HTTP/1.1" 200 None 2026-03-24 01:17:00,189 - lava_callback - DEBUG - Processing node 69c1e38e0efaf21d93106011: baseline-arm64 2026-03-24 01:17:00,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:17:00,857 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:17:00,858 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1e38e0efaf21d93106011/log.txt.gz 2026-03-24 01:17:01,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:17:01,345 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:17:01,347 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1e38e0efaf21d93106011/lava-logs.yaml 2026-03-24 01:17:01,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:17:01,522 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:17:01,524 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1e38e0efaf21d93106011/lava_callback.json.gz 2026-03-24 01:17:01,548 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:17:01,563 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e38e0efaf21d93106011 HTTP/1.1" 200 None 2026-03-24 01:17:01,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:17:01,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-24 01:17:01,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:17:01,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e38e0efaf21d93106011 HTTP/1.1" 200 None 2026-03-24 01:17:01,684 - lava_callback - INFO - Completed processing callback for node 69c1e38e0efaf21d93106011 INFO: 172.21.0.1:41750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41752 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:17:10,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:17:10,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-24 01:17:11,607 - lava_callback - INFO - Received callback for node 69c1e5ba0efaf21d93106377 INFO: 172.21.0.1:34002 - "POST /node/69c1e5ba0efaf21d93106377 HTTP/1.0" 202 Accepted 2026-03-24 01:17:11,614 - lava_callback - INFO - Starting async job submit for node 69c1e5ba0efaf21d93106377 2026-03-24 01:17:11,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:17:11,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e5ba0efaf21d93106377 HTTP/1.1" 200 None 2026-03-24 01:17:11,659 - lava_callback - DEBUG - Processing node 69c1e5ba0efaf21d93106377: baseline-arm64 2026-03-24 01:17:11,854 - lava_callback - WARNING - Failed to upload log 2026-03-24 01:17:11,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:17:12,144 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:17:12,146 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1e5ba0efaf21d93106377/lava-logs.yaml 2026-03-24 01:17:12,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:17:12,261 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:17:12,263 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1e5ba0efaf21d93106377/lava_callback.json.gz 2026-03-24 01:17:12,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:17:12,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e5ba0efaf21d93106377 HTTP/1.1" 200 None 2026-03-24 01:17:12,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:17:12,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-24 01:17:12,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:17:12,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e5ba0efaf21d93106377 HTTP/1.1" 200 None 2026-03-24 01:17:12,431 - lava_callback - INFO - Completed processing callback for node 69c1e5ba0efaf21d93106377 INFO: 172.21.0.1:34006 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34188 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:17:40,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:17:40,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-24 01:17:46,216 - lava_callback - INFO - Received callback for node 69c1e3a20efaf21d9310603f INFO: 172.21.0.1:40148 - "POST /node/69c1e3a20efaf21d9310603f HTTP/1.0" 202 Accepted 2026-03-24 01:17:46,223 - lava_callback - INFO - Starting async job submit for node 69c1e3a20efaf21d9310603f 2026-03-24 01:17:46,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:17:46,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3a20efaf21d9310603f HTTP/1.1" 200 None 2026-03-24 01:17:46,268 - lava_callback - DEBUG - Processing node 69c1e3a20efaf21d9310603f: baseline-nfs-arm64 INFO: 172.21.0.1:40154 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:17:46,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:17:46,929 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:17:46,930 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c1e3a20efaf21d9310603f/log.txt.gz 2026-03-24 01:17:47,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:17:47,426 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:17:47,428 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c1e3a20efaf21d9310603f/lava-logs.yaml 2026-03-24 01:17:47,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:17:47,618 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:17:47,620 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c1e3a20efaf21d9310603f/lava_callback.json.gz 2026-03-24 01:17:47,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:17:47,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3a20efaf21d9310603f HTTP/1.1" 200 None 2026-03-24 01:17:47,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:17:47,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-24 01:17:47,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:17:47,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e3a20efaf21d9310603f HTTP/1.1" 200 None 2026-03-24 01:17:47,765 - lava_callback - INFO - Completed processing callback for node 69c1e3a20efaf21d9310603f INFO: 172.21.0.1:52376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45152 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:18:10,192 - lava_callback - INFO - Received callback for node 69c1e6420efaf21d931063e2 INFO: 172.21.0.1:33532 - "POST /node/69c1e6420efaf21d931063e2 HTTP/1.0" 202 Accepted 2026-03-24 01:18:10,198 - lava_callback - INFO - Starting async job submit for node 69c1e6420efaf21d931063e2 2026-03-24 01:18:10,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:18:10,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e6420efaf21d931063e2 HTTP/1.1" 200 None 2026-03-24 01:18:10,216 - lava_callback - DEBUG - Processing node 69c1e6420efaf21d931063e2: baseline-nfs-arm64 2026-03-24 01:18:10,218 - lava_callback - WARNING - Failed to upload log 2026-03-24 01:18:10,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:18:10,333 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:18:10,334 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c1e6420efaf21d931063e2/lava-logs.yaml 2026-03-24 01:18:10,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:18:10,457 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:18:10,459 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c1e6420efaf21d931063e2/lava_callback.json.gz 2026-03-24 01:18:10,461 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:18:10,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e6420efaf21d931063e2 HTTP/1.1" 200 None 2026-03-24 01:18:10,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:18:10,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-24 01:18:10,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:18:10,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e6420efaf21d931063e2 HTTP/1.1" 200 None 2026-03-24 01:18:10,526 - lava_callback - INFO - Completed processing callback for node 69c1e6420efaf21d931063e2 2026-03-24 01:18:10,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:18:10,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:33538 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51822 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52070 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35124 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57288 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46216 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56620 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49278 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44690 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49298 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:26:48,315 - lava_callback - INFO - Received callback for node 69c1e39c0efaf21d9310602e INFO: 172.21.0.1:49312 - "POST /node/69c1e39c0efaf21d9310602e HTTP/1.0" 202 Accepted 2026-03-24 01:26:48,324 - lava_callback - INFO - Starting async job submit for node 69c1e39c0efaf21d9310602e 2026-03-24 01:26:48,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:26:48,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e39c0efaf21d9310602e HTTP/1.1" 200 None 2026-03-24 01:26:48,385 - lava_callback - DEBUG - Processing node 69c1e39c0efaf21d9310602e: baseline-nfs-arm64 2026-03-24 01:26:49,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:26:49,740 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:26:49,742 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c1e39c0efaf21d9310602e/log.txt.gz 2026-03-24 01:26:50,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:26:50,647 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:26:50,649 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c1e39c0efaf21d9310602e/lava-logs.yaml 2026-03-24 01:26:50,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:26:50,870 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:26:50,872 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c1e39c0efaf21d9310602e/lava_callback.json.gz DEBUG: Setup failed for 69c1e39c0efaf21d9310602e. Transitting job node result: incomplete -> fail 2026-03-24 01:26:50,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:26:50,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e39c0efaf21d9310602e HTTP/1.1" 200 None 2026-03-24 01:26:50,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:26:51,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-24 01:26:51,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:26:51,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e39c0efaf21d9310602e HTTP/1.1" 200 None 2026-03-24 01:26:51,141 - lava_callback - INFO - Completed processing callback for node 69c1e39c0efaf21d9310602e INFO: 172.21.0.1:44080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33768 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:27:10,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:27:10,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:38656 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46318 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:27:55,678 - lava_callback - INFO - Received callback for node 69bf4597f1004e82292e3bba INFO: 172.21.0.1:46614 - "POST /node/69bf4597f1004e82292e3bba HTTP/1.0" 202 Accepted 2026-03-24 01:27:55,683 - lava_callback - INFO - Starting async job submit for node 69bf4597f1004e82292e3bba 2026-03-24 01:27:55,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:27:55,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf4597f1004e82292e3bba HTTP/1.1" 200 None 2026-03-24 01:27:55,710 - lava_callback - DEBUG - Processing node 69bf4597f1004e82292e3bba: baseline-arm64-baylibre 2026-03-24 01:27:55,710 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf4597f1004e82292e3bba 2026-03-24 01:27:55,711 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:27:55,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf4597f1004e82292e3bba HTTP/1.1" 200 None 2026-03-24 01:27:55,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:27:55,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21b717e5cb936602bc2b HTTP/1.1" 200 None 2026-03-24 01:27:55,743 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:27:55,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf4597f1004e82292e3bba HTTP/1.1" 200 None 2026-03-24 01:27:55,770 - lava_callback - ERROR - Error processing callback for node 69bf4597f1004e82292e3bba: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52988 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52422 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34036 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36764 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40248 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-24 01:32:15,814 - lava_callback - INFO - Received callback for node 69bf4699f1004e82292e3c1d INFO: 172.21.0.1:40262 - "POST /node/69bf4699f1004e82292e3c1d HTTP/1.0" 202 Accepted 2026-03-24 01:32:15,820 - lava_callback - INFO - Starting async job submit for node 69bf4699f1004e82292e3c1d 2026-03-24 01:32:15,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:32:15,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf4699f1004e82292e3c1d HTTP/1.1" 200 None 2026-03-24 01:32:15,835 - lava_callback - DEBUG - Processing node 69bf4699f1004e82292e3c1d: baseline-arm64-baylibre 2026-03-24 01:32:15,835 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf4699f1004e82292e3c1d 2026-03-24 01:32:15,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:32:15,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf4699f1004e82292e3c1d HTTP/1.1" 200 None 2026-03-24 01:32:15,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:32:15,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21b717e5cb936602bc2b HTTP/1.1" 200 None 2026-03-24 01:32:15,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:32:15,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf4699f1004e82292e3c1d HTTP/1.1" 200 None 2026-03-24 01:32:15,911 - lava_callback - ERROR - Error processing callback for node 69bf4699f1004e82292e3c1d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51166 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59592 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47410 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:34:36,290 - lava_callback - INFO - Received callback for node 69bf4728f1004e82292e3c3e INFO: 172.21.0.1:43322 - "POST /node/69bf4728f1004e82292e3c3e HTTP/1.0" 202 Accepted 2026-03-24 01:34:36,296 - lava_callback - INFO - Starting async job submit for node 69bf4728f1004e82292e3c3e 2026-03-24 01:34:36,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:34:36,310 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf4728f1004e82292e3c3e HTTP/1.1" 200 None 2026-03-24 01:34:36,311 - lava_callback - DEBUG - Processing node 69bf4728f1004e82292e3c3e: baseline-arm64-baylibre 2026-03-24 01:34:36,311 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf4728f1004e82292e3c3e 2026-03-24 01:34:36,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:34:36,325 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf4728f1004e82292e3c3e HTTP/1.1" 200 None 2026-03-24 01:34:36,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:34:36,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21b717e5cb936602bc2b HTTP/1.1" 200 None 2026-03-24 01:34:36,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:34:36,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf4728f1004e82292e3c3e HTTP/1.1" 200 None 2026-03-24 01:34:36,367 - lava_callback - ERROR - Error processing callback for node 69bf4728f1004e82292e3c3e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55352 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:34:49,346 - lava_callback - INFO - Received callback for node 69c1ea070efaf21d93106687 INFO: 172.21.0.1:55364 - "POST /node/69c1ea070efaf21d93106687 HTTP/1.0" 202 Accepted 2026-03-24 01:34:49,353 - lava_callback - INFO - Starting async job submit for node 69c1ea070efaf21d93106687 2026-03-24 01:34:49,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:34:49,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1ea070efaf21d93106687 HTTP/1.1" 200 None 2026-03-24 01:34:49,399 - lava_callback - DEBUG - Processing node 69c1ea070efaf21d93106687: baseline-arm64 2026-03-24 01:34:49,534 - lava_callback - WARNING - Failed to upload log 2026-03-24 01:34:49,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:34:49,835 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:34:49,837 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1ea070efaf21d93106687/lava-logs.yaml 2026-03-24 01:34:49,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:34:49,968 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:34:49,969 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1ea070efaf21d93106687/lava_callback.json.gz 2026-03-24 01:34:49,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:34:50,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1ea070efaf21d93106687 HTTP/1.1" 200 None 2026-03-24 01:34:50,015 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:34:50,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-24 01:34:50,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:34:50,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1ea070efaf21d93106687 HTTP/1.1" 200 None 2026-03-24 01:34:50,138 - lava_callback - INFO - Completed processing callback for node 69c1ea070efaf21d93106687 INFO: 172.21.0.1:33802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55650 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:35:10,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:35:10,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:55656 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60570 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47664 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57196 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39620 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:38:37,464 - lava_callback - INFO - Received callback for node 69bf480bf1004e82292e3c5a INFO: 172.21.0.1:39622 - "POST /node/69bf480bf1004e82292e3c5a HTTP/1.0" 202 Accepted 2026-03-24 01:38:37,473 - lava_callback - INFO - Starting async job submit for node 69bf480bf1004e82292e3c5a 2026-03-24 01:38:37,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:38:37,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf480bf1004e82292e3c5a HTTP/1.1" 200 None 2026-03-24 01:38:37,495 - lava_callback - DEBUG - Processing node 69bf480bf1004e82292e3c5a: baseline-arm64-clabbe 2026-03-24 01:38:37,495 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf480bf1004e82292e3c5a 2026-03-24 01:38:37,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:38:37,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf480bf1004e82292e3c5a HTTP/1.1" 200 None 2026-03-24 01:38:37,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:38:37,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21b717e5cb936602bc2b HTTP/1.1" 200 None 2026-03-24 01:38:37,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:38:37,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf480bf1004e82292e3c5a HTTP/1.1" 200 None 2026-03-24 01:38:37,577 - lava_callback - ERROR - Error processing callback for node 69bf480bf1004e82292e3c5a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42656 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:38:49,640 - lava_callback - INFO - Received callback for node 69c1eadf0efaf21d93106702 INFO: 172.21.0.1:42660 - "POST /node/69c1eadf0efaf21d93106702 HTTP/1.0" 202 Accepted 2026-03-24 01:38:49,647 - lava_callback - INFO - Starting async job submit for node 69c1eadf0efaf21d93106702 2026-03-24 01:38:49,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:38:49,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1eadf0efaf21d93106702 HTTP/1.1" 200 None 2026-03-24 01:38:49,714 - lava_callback - DEBUG - Processing node 69c1eadf0efaf21d93106702: baseline-arm64 2026-03-24 01:38:50,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:38:50,443 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:38:50,445 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1eadf0efaf21d93106702/log.txt.gz 2026-03-24 01:38:50,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:38:51,150 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:38:51,153 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1eadf0efaf21d93106702/lava-logs.yaml 2026-03-24 01:38:51,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:38:51,392 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:38:51,394 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1eadf0efaf21d93106702/lava_callback.json.gz 2026-03-24 01:38:51,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:38:51,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1eadf0efaf21d93106702 HTTP/1.1" 200 None 2026-03-24 01:38:51,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:38:51,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-24 01:38:51,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:38:51,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1eadf0efaf21d93106702 HTTP/1.1" 200 None 2026-03-24 01:38:51,576 - lava_callback - INFO - Completed processing callback for node 69c1eadf0efaf21d93106702 INFO: 172.21.0.1:33704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43648 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:39:10,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:39:10,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:43654 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44514 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41358 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47462 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:41:36,480 - lava_callback - INFO - Received callback for node 69bf48baf1004e82292e3c8c INFO: 172.21.0.1:60182 - "POST /node/69bf48baf1004e82292e3c8c HTTP/1.0" 202 Accepted 2026-03-24 01:41:36,485 - lava_callback - INFO - Starting async job submit for node 69bf48baf1004e82292e3c8c 2026-03-24 01:41:36,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:41:36,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf48baf1004e82292e3c8c HTTP/1.1" 200 None 2026-03-24 01:41:36,501 - lava_callback - DEBUG - Processing node 69bf48baf1004e82292e3c8c: baseline-arm64-baylibre 2026-03-24 01:41:36,501 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf48baf1004e82292e3c8c 2026-03-24 01:41:36,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:41:36,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf48baf1004e82292e3c8c HTTP/1.1" 200 None 2026-03-24 01:41:36,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:41:36,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21b717e5cb936602bc2b HTTP/1.1" 200 None 2026-03-24 01:41:36,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:41:36,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf48baf1004e82292e3c8c HTTP/1.1" 200 None 2026-03-24 01:41:36,554 - lava_callback - ERROR - Error processing callback for node 69bf48baf1004e82292e3c8c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60194 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:41:42,388 - lava_callback - INFO - Received callback for node 69c1eb8f0efaf21d93106775 INFO: 172.21.0.1:43978 - "POST /node/69c1eb8f0efaf21d93106775 HTTP/1.0" 202 Accepted 2026-03-24 01:41:42,395 - lava_callback - INFO - Starting async job submit for node 69c1eb8f0efaf21d93106775 2026-03-24 01:41:42,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:41:42,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1eb8f0efaf21d93106775 HTTP/1.1" 200 None 2026-03-24 01:41:42,448 - lava_callback - DEBUG - Processing node 69c1eb8f0efaf21d93106775: baseline-arm64 2026-03-24 01:41:42,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:41:42,794 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:41:42,796 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1eb8f0efaf21d93106775/log.txt.gz 2026-03-24 01:41:42,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:41:43,073 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:41:43,075 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1eb8f0efaf21d93106775/lava-logs.yaml 2026-03-24 01:41:43,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:41:43,221 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:41:43,222 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1eb8f0efaf21d93106775/lava_callback.json.gz 2026-03-24 01:41:43,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:41:43,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1eb8f0efaf21d93106775 HTTP/1.1" 200 None 2026-03-24 01:41:43,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:41:43,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-24 01:41:43,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:41:43,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1eb8f0efaf21d93106775 HTTP/1.1" 200 None 2026-03-24 01:41:43,376 - lava_callback - INFO - Completed processing callback for node 69c1eb8f0efaf21d93106775 INFO: 172.21.0.1:43992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51734 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:42:10,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:42:10,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:51740 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54086 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:42:57,660 - lava_callback - INFO - Received callback for node 69bf490cf1004e82292e3c98 INFO: 172.21.0.1:54102 - "POST /node/69bf490cf1004e82292e3c98 HTTP/1.0" 202 Accepted 2026-03-24 01:42:57,666 - lava_callback - INFO - Starting async job submit for node 69bf490cf1004e82292e3c98 2026-03-24 01:42:57,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:42:57,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf490cf1004e82292e3c98 HTTP/1.1" 200 None 2026-03-24 01:42:57,694 - lava_callback - DEBUG - Processing node 69bf490cf1004e82292e3c98: baseline-arm64-clabbe 2026-03-24 01:42:57,694 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf490cf1004e82292e3c98 2026-03-24 01:42:57,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:42:57,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf490cf1004e82292e3c98 HTTP/1.1" 200 None 2026-03-24 01:42:57,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:42:57,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21b717e5cb936602bc2b HTTP/1.1" 200 None 2026-03-24 01:42:57,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:42:57,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf490cf1004e82292e3c98 HTTP/1.1" 200 None 2026-03-24 01:42:57,752 - lava_callback - ERROR - Error processing callback for node 69bf490cf1004e82292e3c98: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58056 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58072 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:43:18,082 - lava_callback - INFO - Received callback for node 69c1ea620efaf21d931066c0 INFO: 172.21.0.1:58082 - "POST /node/69c1ea620efaf21d931066c0 HTTP/1.0" 202 Accepted 2026-03-24 01:43:18,090 - lava_callback - INFO - Starting async job submit for node 69c1ea620efaf21d931066c0 2026-03-24 01:43:18,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:43:18,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1ea620efaf21d931066c0 HTTP/1.1" 200 None 2026-03-24 01:43:18,151 - lava_callback - DEBUG - Processing node 69c1ea620efaf21d931066c0: baseline-arm64 2026-03-24 01:43:18,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:43:18,697 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:43:18,698 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1ea620efaf21d931066c0/log.txt.gz 2026-03-24 01:43:18,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:43:19,178 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:43:19,180 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1ea620efaf21d931066c0/lava-logs.yaml 2026-03-24 01:43:19,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:43:19,363 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:43:19,365 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1ea620efaf21d931066c0/lava_callback.json.gz DEBUG: Setup failed for 69c1ea620efaf21d931066c0. Transitting job node result: incomplete -> fail 2026-03-24 01:43:19,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:43:19,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1ea620efaf21d931066c0 HTTP/1.1" 200 None 2026-03-24 01:43:19,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:43:19,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-24 01:43:19,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:43:19,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1ea620efaf21d931066c0 HTTP/1.1" 200 None 2026-03-24 01:43:19,587 - lava_callback - INFO - Completed processing callback for node 69c1ea620efaf21d931066c0 INFO: 172.21.0.1:37338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33784 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:43:40,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:43:40,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:43666 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:43:51,572 - lava_callback - INFO - Received callback for node 69c1ec460efaf21d931067e1 INFO: 172.21.0.1:37914 - "POST /node/69c1ec460efaf21d931067e1 HTTP/1.0" 202 Accepted 2026-03-24 01:43:51,578 - lava_callback - INFO - Starting async job submit for node 69c1ec460efaf21d931067e1 2026-03-24 01:43:51,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:43:51,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1ec460efaf21d931067e1 HTTP/1.1" 200 None 2026-03-24 01:43:51,598 - lava_callback - DEBUG - Processing node 69c1ec460efaf21d931067e1: baseline-nfs-arm64 2026-03-24 01:43:51,600 - lava_callback - WARNING - Failed to upload log 2026-03-24 01:43:51,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:43:51,727 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:43:51,729 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c1ec460efaf21d931067e1/lava-logs.yaml 2026-03-24 01:43:51,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:43:51,870 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:43:51,872 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c1ec460efaf21d931067e1/lava_callback.json.gz 2026-03-24 01:43:51,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:43:51,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1ec460efaf21d931067e1 HTTP/1.1" 200 None 2026-03-24 01:43:51,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:43:51,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-24 01:43:51,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:43:51,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1ec460efaf21d931067e1 HTTP/1.1" 200 None 2026-03-24 01:43:51,943 - lava_callback - INFO - Completed processing callback for node 69c1ec460efaf21d931067e1 INFO: 172.21.0.1:37916 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:43:57,658 - lava_callback - INFO - Received callback for node 69bf4957f1004e82292e3c9e INFO: 172.21.0.1:37926 - "POST /node/69bf4957f1004e82292e3c9e HTTP/1.0" 202 Accepted 2026-03-24 01:43:57,664 - lava_callback - INFO - Starting async job submit for node 69bf4957f1004e82292e3c9e 2026-03-24 01:43:57,665 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:43:57,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf4957f1004e82292e3c9e HTTP/1.1" 200 None 2026-03-24 01:43:57,680 - lava_callback - DEBUG - Processing node 69bf4957f1004e82292e3c9e: baseline-arm64-clabbe 2026-03-24 01:43:57,680 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf4957f1004e82292e3c9e 2026-03-24 01:43:57,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:43:57,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf4957f1004e82292e3c9e HTTP/1.1" 200 None 2026-03-24 01:43:57,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:43:57,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21b717e5cb936602bc2b HTTP/1.1" 200 None 2026-03-24 01:43:57,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:43:57,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf4957f1004e82292e3c9e HTTP/1.1" 200 None 2026-03-24 01:43:57,730 - lava_callback - ERROR - Error processing callback for node 69bf4957f1004e82292e3c9e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38306 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:44:10,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:44:10,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:38318 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41726 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.21.0.1:44226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58588 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-24 01:45:16,495 - lava_callback - INFO - Received callback for node 69bf4998f1004e82292e3ca4 INFO: 172.21.0.1:58596 - "POST /node/69bf4998f1004e82292e3ca4 HTTP/1.0" 202 Accepted 2026-03-24 01:45:16,501 - lava_callback - INFO - Starting async job submit for node 69bf4998f1004e82292e3ca4 2026-03-24 01:45:16,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:45:16,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf4998f1004e82292e3ca4 HTTP/1.1" 200 None 2026-03-24 01:45:16,516 - lava_callback - DEBUG - Processing node 69bf4998f1004e82292e3ca4: baseline-arm64-baylibre 2026-03-24 01:45:16,516 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf4998f1004e82292e3ca4 2026-03-24 01:45:16,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:45:16,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf4998f1004e82292e3ca4 HTTP/1.1" 200 None 2026-03-24 01:45:16,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:45:16,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21b717e5cb936602bc2b HTTP/1.1" 200 None 2026-03-24 01:45:16,548 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:58610 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:45:16,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf4998f1004e82292e3ca4 HTTP/1.1" 200 None 2026-03-24 01:45:16,578 - lava_callback - ERROR - Error processing callback for node 69bf4998f1004e82292e3ca4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51176 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50418 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:46:37,876 - lava_callback - INFO - Received callback for node 69bf49ebf1004e82292e3cae INFO: 172.21.0.1:50426 - "POST /node/69bf49ebf1004e82292e3cae HTTP/1.0" 202 Accepted 2026-03-24 01:46:37,883 - lava_callback - INFO - Starting async job submit for node 69bf49ebf1004e82292e3cae 2026-03-24 01:46:37,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:46:37,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf49ebf1004e82292e3cae HTTP/1.1" 200 None 2026-03-24 01:46:37,898 - lava_callback - DEBUG - Processing node 69bf49ebf1004e82292e3cae: baseline-arm64-clabbe 2026-03-24 01:46:37,898 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf49ebf1004e82292e3cae 2026-03-24 01:46:37,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:46:37,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf49ebf1004e82292e3cae HTTP/1.1" 200 None 2026-03-24 01:46:37,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:46:37,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21b717e5cb936602bc2b HTTP/1.1" 200 None 2026-03-24 01:46:37,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:46:37,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf49ebf1004e82292e3cae HTTP/1.1" 200 None 2026-03-24 01:46:37,948 - lava_callback - ERROR - Error processing callback for node 69bf49ebf1004e82292e3cae: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55224 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58984 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59102 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:48:56,748 - lava_callback - INFO - Received callback for node 69bf4a79f1004e82292e3cd2 INFO: 172.21.0.1:59104 - "POST /node/69bf4a79f1004e82292e3cd2 HTTP/1.0" 202 Accepted 2026-03-24 01:48:56,754 - lava_callback - INFO - Starting async job submit for node 69bf4a79f1004e82292e3cd2 2026-03-24 01:48:56,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:48:56,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf4a79f1004e82292e3cd2 HTTP/1.1" 200 None 2026-03-24 01:48:56,769 - lava_callback - DEBUG - Processing node 69bf4a79f1004e82292e3cd2: baseline-arm64-baylibre 2026-03-24 01:48:56,769 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf4a79f1004e82292e3cd2 2026-03-24 01:48:56,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:48:56,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf4a79f1004e82292e3cd2 HTTP/1.1" 200 None 2026-03-24 01:48:56,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:48:56,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21b717e5cb936602bc2b HTTP/1.1" 200 None 2026-03-24 01:48:56,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:48:56,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf4a79f1004e82292e3cd2 HTTP/1.1" 200 None 2026-03-24 01:48:56,853 - lava_callback - ERROR - Error processing callback for node 69bf4a79f1004e82292e3cd2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52708 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60126 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35662 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:51:04,820 - lava_callback - INFO - Received callback for node 69c1eab30efaf21d931066f2 INFO: 172.21.0.1:40702 - "POST /node/69c1eab30efaf21d931066f2 HTTP/1.0" 202 Accepted 2026-03-24 01:51:04,827 - lava_callback - INFO - Starting async job submit for node 69c1eab30efaf21d931066f2 2026-03-24 01:51:04,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:51:04,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1eab30efaf21d931066f2 HTTP/1.1" 200 None 2026-03-24 01:51:04,891 - lava_callback - DEBUG - Processing node 69c1eab30efaf21d931066f2: baseline-arm 2026-03-24 01:51:05,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:51:05,480 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:51:05,482 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c1eab30efaf21d931066f2/log.txt.gz 2026-03-24 01:51:05,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:51:06,042 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:51:06,044 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c1eab30efaf21d931066f2/lava-logs.yaml 2026-03-24 01:51:06,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:51:06,247 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:51:06,249 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c1eab30efaf21d931066f2/lava_callback.json.gz DEBUG: Setup failed for 69c1eab30efaf21d931066f2. Transitting job node result: incomplete -> fail 2026-03-24 01:51:06,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:51:06,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1eab30efaf21d931066f2 HTTP/1.1" 200 None 2026-03-24 01:51:06,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:51:06,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-24 01:51:06,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:51:06,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1eab30efaf21d931066f2 HTTP/1.1" 200 None INFO: 172.21.0.1:40712 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:51:06,531 - lava_callback - INFO - Completed processing callback for node 69c1eab30efaf21d931066f2 INFO: 172.21.0.1:41016 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:51:10,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:51:11,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:41032 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58256 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32968 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:52:38,055 - lava_callback - INFO - Received callback for node 69bf4b5cf1004e82292e3cfc INFO: 172.21.0.1:32984 - "POST /node/69bf4b5cf1004e82292e3cfc HTTP/1.0" 202 Accepted 2026-03-24 01:52:38,060 - lava_callback - INFO - Starting async job submit for node 69bf4b5cf1004e82292e3cfc 2026-03-24 01:52:38,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:52:38,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf4b5cf1004e82292e3cfc HTTP/1.1" 200 None 2026-03-24 01:52:38,077 - lava_callback - DEBUG - Processing node 69bf4b5cf1004e82292e3cfc: baseline-arm64-clabbe 2026-03-24 01:52:38,077 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf4b5cf1004e82292e3cfc 2026-03-24 01:52:38,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:52:38,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf4b5cf1004e82292e3cfc HTTP/1.1" 200 None 2026-03-24 01:52:38,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:52:38,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21b717e5cb936602bc2b HTTP/1.1" 200 None 2026-03-24 01:52:38,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:52:38,129 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf4b5cf1004e82292e3cfc HTTP/1.1" 200 None 2026-03-24 01:52:38,131 - lava_callback - ERROR - Error processing callback for node 69bf4b5cf1004e82292e3cfc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55102 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:53:10,214 - lava_callback - INFO - Received callback for node 69c1eb3b0efaf21d93106731 INFO: 172.21.0.1:55112 - "POST /node/69c1eb3b0efaf21d93106731 HTTP/1.0" 202 Accepted 2026-03-24 01:53:10,221 - lava_callback - INFO - Starting async job submit for node 69c1eb3b0efaf21d93106731 2026-03-24 01:53:10,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:53:10,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1eb3b0efaf21d93106731 HTTP/1.1" 200 None 2026-03-24 01:53:10,260 - lava_callback - DEBUG - Processing node 69c1eb3b0efaf21d93106731: baseline-arm64 2026-03-24 01:53:10,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:53:10,923 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:53:10,925 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1eb3b0efaf21d93106731/log.txt.gz 2026-03-24 01:53:11,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:53:11,452 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:53:11,455 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1eb3b0efaf21d93106731/lava-logs.yaml 2026-03-24 01:53:11,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:53:11,632 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:53:11,634 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1eb3b0efaf21d93106731/lava_callback.json.gz 2026-03-24 01:53:11,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:53:11,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1eb3b0efaf21d93106731 HTTP/1.1" 200 None 2026-03-24 01:53:11,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:53:11,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-24 01:53:11,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:53:11,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1eb3b0efaf21d93106731 HTTP/1.1" 200 None 2026-03-24 01:53:11,798 - lava_callback - INFO - Completed processing callback for node 69c1eb3b0efaf21d93106731 INFO: 172.21.0.1:55116 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49204 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:53:36,917 - lava_callback - INFO - Received callback for node 69bf4b9ef1004e82292e3d09 INFO: 172.21.0.1:49214 - "POST /node/69bf4b9ef1004e82292e3d09 HTTP/1.0" 202 Accepted 2026-03-24 01:53:36,923 - lava_callback - INFO - Starting async job submit for node 69bf4b9ef1004e82292e3d09 2026-03-24 01:53:36,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:53:36,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf4b9ef1004e82292e3d09 HTTP/1.1" 200 None 2026-03-24 01:53:36,939 - lava_callback - DEBUG - Processing node 69bf4b9ef1004e82292e3d09: baseline-arm64-baylibre 2026-03-24 01:53:36,940 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf4b9ef1004e82292e3d09 2026-03-24 01:53:36,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:53:36,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf4b9ef1004e82292e3d09 HTTP/1.1" 200 None 2026-03-24 01:53:36,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:53:36,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-24 01:53:36,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:53:36,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf4b9ef1004e82292e3d09 HTTP/1.1" 200 None 2026-03-24 01:53:36,998 - lava_callback - ERROR - Error processing callback for node 69bf4b9ef1004e82292e3d09: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-24 01:53:41,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:53:41,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:34274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38340 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46780 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:54:57,014 - lava_callback - INFO - Received callback for node 69bf4be8f1004e82292e3d0f INFO: 172.21.0.1:46794 - "POST /node/69bf4be8f1004e82292e3d0f HTTP/1.0" 202 Accepted 2026-03-24 01:54:57,019 - lava_callback - INFO - Starting async job submit for node 69bf4be8f1004e82292e3d0f 2026-03-24 01:54:57,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:54:57,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf4be8f1004e82292e3d0f HTTP/1.1" 200 None 2026-03-24 01:54:57,035 - lava_callback - DEBUG - Processing node 69bf4be8f1004e82292e3d0f: baseline-arm64-baylibre 2026-03-24 01:54:57,035 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf4be8f1004e82292e3d0f 2026-03-24 01:54:57,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:54:57,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf4be8f1004e82292e3d0f HTTP/1.1" 200 None 2026-03-24 01:54:57,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:54:57,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-24 01:54:57,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:54:57,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf4be8f1004e82292e3d0f HTTP/1.1" 200 None 2026-03-24 01:54:57,094 - lava_callback - ERROR - Error processing callback for node 69bf4be8f1004e82292e3d0f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34812 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52560 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:55:40,869 - lava_callback - INFO - Received callback for node 69c1e1bd0efaf21d93105f2e INFO: 172.21.0.1:50086 - "POST /node/69c1e1bd0efaf21d93105f2e HTTP/1.0" 202 Accepted 2026-03-24 01:55:40,917 - lava_callback - INFO - Starting async job submit for node 69c1e1bd0efaf21d93105f2e 2026-03-24 01:55:40,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:55:40,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e1bd0efaf21d93105f2e HTTP/1.1" 200 None 2026-03-24 01:55:40,955 - lava_callback - DEBUG - Processing node 69c1e1bd0efaf21d93105f2e: baseline-arm-pengutronix 2026-03-24 01:55:41,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:55:41,206 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:55:41,208 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c1e1bd0efaf21d93105f2e/log.txt.gz 2026-03-24 01:55:41,266 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:55:41,454 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:55:41,456 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c1e1bd0efaf21d93105f2e/lava-logs.yaml 2026-03-24 01:55:41,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:55:41,570 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:55:41,572 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c1e1bd0efaf21d93105f2e/lava_callback.json.gz DEBUG: Setup failed for 69c1e1bd0efaf21d93105f2e. Transitting job node result: incomplete -> fail 2026-03-24 01:55:41,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:55:41,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e1bd0efaf21d93105f2e HTTP/1.1" 200 None 2026-03-24 01:55:41,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:55:41,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-24 01:55:41,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:55:41,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e1bd0efaf21d93105f2e HTTP/1.1" 200 None 2026-03-24 01:55:41,737 - lava_callback - INFO - Completed processing callback for node 69c1e1bd0efaf21d93105f2e INFO: 172.21.0.1:50094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59658 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:56:11,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:56:11,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:59660 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59670 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:56:18,023 - lava_callback - INFO - Received callback for node 69c1ebf00efaf21d931067a9 INFO: 172.21.0.1:59672 - "POST /node/69c1ebf00efaf21d931067a9 HTTP/1.0" 202 Accepted 2026-03-24 01:56:18,030 - lava_callback - INFO - Starting async job submit for node 69c1ebf00efaf21d931067a9 2026-03-24 01:56:18,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:56:18,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1ebf00efaf21d931067a9 HTTP/1.1" 200 None 2026-03-24 01:56:18,071 - lava_callback - DEBUG - Processing node 69c1ebf00efaf21d931067a9: baseline-nfs-arm64 2026-03-24 01:56:18,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:56:18,746 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:56:18,748 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c1ebf00efaf21d931067a9/log.txt.gz 2026-03-24 01:56:19,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:56:19,225 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:56:19,227 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c1ebf00efaf21d931067a9/lava-logs.yaml 2026-03-24 01:56:19,239 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 01:56:19,408 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 01:56:19,410 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c1ebf00efaf21d931067a9/lava_callback.json.gz 2026-03-24 01:56:19,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:56:19,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1ebf00efaf21d931067a9 HTTP/1.1" 200 None 2026-03-24 01:56:19,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:56:19,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-24 01:56:19,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:56:19,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1ebf00efaf21d931067a9 HTTP/1.1" 200 None 2026-03-24 01:56:19,589 - lava_callback - INFO - Completed processing callback for node 69c1ebf00efaf21d931067a9 INFO: 172.21.0.1:36870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41348 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:56:41,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:56:41,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:49978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35838 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51386 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:57:37,257 - lava_callback - INFO - Received callback for node 69bf4c7df1004e82292e3d23 INFO: 172.21.0.1:51400 - "POST /node/69bf4c7df1004e82292e3d23 HTTP/1.0" 202 Accepted 2026-03-24 01:57:37,265 - lava_callback - INFO - Starting async job submit for node 69bf4c7df1004e82292e3d23 2026-03-24 01:57:37,266 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:57:37,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf4c7df1004e82292e3d23 HTTP/1.1" 200 None 2026-03-24 01:57:37,281 - lava_callback - DEBUG - Processing node 69bf4c7df1004e82292e3d23: baseline-arm64-baylibre 2026-03-24 01:57:37,281 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf4c7df1004e82292e3d23 2026-03-24 01:57:37,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:57:37,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf4c7df1004e82292e3d23 HTTP/1.1" 200 None 2026-03-24 01:57:37,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:57:37,310 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-24 01:57:37,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:57:37,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf4c7df1004e82292e3d23 HTTP/1.1" 200 None 2026-03-24 01:57:37,369 - lava_callback - ERROR - Error processing callback for node 69bf4c7df1004e82292e3d23: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41664 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38512 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:58:37,180 - lava_callback - INFO - Received callback for node 69bf4cc8f1004e82292e3d32 INFO: 172.21.0.1:38524 - "POST /node/69bf4cc8f1004e82292e3d32 HTTP/1.0" 202 Accepted 2026-03-24 01:58:37,185 - lava_callback - INFO - Starting async job submit for node 69bf4cc8f1004e82292e3d32 2026-03-24 01:58:37,186 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:58:37,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf4cc8f1004e82292e3d32 HTTP/1.1" 200 None 2026-03-24 01:58:37,212 - lava_callback - DEBUG - Processing node 69bf4cc8f1004e82292e3d32: baseline-arm64-baylibre 2026-03-24 01:58:37,213 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf4cc8f1004e82292e3d32 2026-03-24 01:58:37,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:58:37,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf4cc8f1004e82292e3d32 HTTP/1.1" 200 None 2026-03-24 01:58:37,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:58:37,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-24 01:58:37,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:58:37,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf4cc8f1004e82292e3d32 HTTP/1.1" 200 None 2026-03-24 01:58:37,265 - lava_callback - ERROR - Error processing callback for node 69bf4cc8f1004e82292e3d32: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47790 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32806 - "GET / HTTP/1.0" 200 OK 2026-03-24 01:59:58,323 - lava_callback - INFO - Received callback for node 69bf4d18f1004e82292e3d3e INFO: 172.21.0.1:32810 - "POST /node/69bf4d18f1004e82292e3d3e HTTP/1.0" 202 Accepted 2026-03-24 01:59:58,328 - lava_callback - INFO - Starting async job submit for node 69bf4d18f1004e82292e3d3e 2026-03-24 01:59:58,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:59:58,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf4d18f1004e82292e3d3e HTTP/1.1" 200 None 2026-03-24 01:59:58,344 - lava_callback - DEBUG - Processing node 69bf4d18f1004e82292e3d3e: baseline-arm64-clabbe 2026-03-24 01:59:58,344 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf4d18f1004e82292e3d3e 2026-03-24 01:59:58,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:59:58,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf4d18f1004e82292e3d3e HTTP/1.1" 200 None 2026-03-24 01:59:58,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:59:58,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-24 01:59:58,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 01:59:58,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf4d18f1004e82292e3d3e HTTP/1.1" 200 None 2026-03-24 01:59:58,400 - lava_callback - ERROR - Error processing callback for node 69bf4d18f1004e82292e3d3e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32774 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:32784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44524 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44530 - "GET / HTTP/1.0" 200 OK 2026-03-24 02:01:18,337 - lava_callback - INFO - Received callback for node 69bf4d66f1004e82292e3d4b INFO: 172.21.0.1:44540 - "POST /node/69bf4d66f1004e82292e3d4b HTTP/1.0" 202 Accepted 2026-03-24 02:01:18,343 - lava_callback - INFO - Starting async job submit for node 69bf4d66f1004e82292e3d4b 2026-03-24 02:01:18,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:01:18,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf4d66f1004e82292e3d4b HTTP/1.1" 200 None 2026-03-24 02:01:18,360 - lava_callback - DEBUG - Processing node 69bf4d66f1004e82292e3d4b: baseline-arm64-clabbe 2026-03-24 02:01:18,360 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf4d66f1004e82292e3d4b 2026-03-24 02:01:18,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:01:18,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf4d66f1004e82292e3d4b HTTP/1.1" 200 None 2026-03-24 02:01:18,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:01:18,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-24 02:01:18,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:01:18,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf4d66f1004e82292e3d4b HTTP/1.1" 200 None 2026-03-24 02:01:18,436 - lava_callback - ERROR - Error processing callback for node 69bf4d66f1004e82292e3d4b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53240 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40482 - "GET / HTTP/1.0" 200 OK 2026-03-24 02:02:38,405 - lava_callback - INFO - Received callback for node 69bf4db2f1004e82292e3d5d INFO: 172.21.0.1:40492 - "POST /node/69bf4db2f1004e82292e3d5d HTTP/1.0" 202 Accepted 2026-03-24 02:02:38,410 - lava_callback - INFO - Starting async job submit for node 69bf4db2f1004e82292e3d5d 2026-03-24 02:02:38,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:02:38,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf4db2f1004e82292e3d5d HTTP/1.1" 200 None 2026-03-24 02:02:38,429 - lava_callback - DEBUG - Processing node 69bf4db2f1004e82292e3d5d: baseline-arm64-clabbe 2026-03-24 02:02:38,429 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf4db2f1004e82292e3d5d 2026-03-24 02:02:38,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:02:38,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf4db2f1004e82292e3d5d HTTP/1.1" 200 None 2026-03-24 02:02:38,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:02:38,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-24 02:02:38,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:02:38,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf4db2f1004e82292e3d5d HTTP/1.1" 200 None 2026-03-24 02:02:38,511 - lava_callback - ERROR - Error processing callback for node 69bf4db2f1004e82292e3d5d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57720 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34012 - "GET / HTTP/1.0" 200 OK 2026-03-24 02:03:58,407 - lava_callback - INFO - Received callback for node 69bf4dfcf1004e82292e3d6e INFO: 172.21.0.1:34022 - "POST /node/69bf4dfcf1004e82292e3d6e HTTP/1.0" 202 Accepted 2026-03-24 02:03:58,413 - lava_callback - INFO - Starting async job submit for node 69bf4dfcf1004e82292e3d6e 2026-03-24 02:03:58,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:03:58,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf4dfcf1004e82292e3d6e HTTP/1.1" 200 None 2026-03-24 02:03:58,429 - lava_callback - DEBUG - Processing node 69bf4dfcf1004e82292e3d6e: baseline-arm64-clabbe 2026-03-24 02:03:58,429 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf4dfcf1004e82292e3d6e 2026-03-24 02:03:58,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:03:58,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf4dfcf1004e82292e3d6e HTTP/1.1" 200 None 2026-03-24 02:03:58,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:03:58,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-24 02:03:58,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:03:58,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf4dfcf1004e82292e3d6e HTTP/1.1" 200 None 2026-03-24 02:03:58,493 - lava_callback - ERROR - Error processing callback for node 69bf4dfcf1004e82292e3d6e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35402 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35604 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35610 - "GET / HTTP/1.0" 200 OK 2026-03-24 02:05:18,421 - lava_callback - INFO - Received callback for node 69bf4e48f1004e82292e3d79 INFO: 172.21.0.1:35618 - "POST /node/69bf4e48f1004e82292e3d79 HTTP/1.0" 202 Accepted 2026-03-24 02:05:18,427 - lava_callback - INFO - Starting async job submit for node 69bf4e48f1004e82292e3d79 2026-03-24 02:05:18,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:05:18,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf4e48f1004e82292e3d79 HTTP/1.1" 200 None 2026-03-24 02:05:18,443 - lava_callback - DEBUG - Processing node 69bf4e48f1004e82292e3d79: baseline-arm64-clabbe 2026-03-24 02:05:18,443 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf4e48f1004e82292e3d79 2026-03-24 02:05:18,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:05:18,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf4e48f1004e82292e3d79 HTTP/1.1" 200 None 2026-03-24 02:05:18,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:05:18,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-24 02:05:18,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:05:18,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf4e48f1004e82292e3d79 HTTP/1.1" 200 None 2026-03-24 02:05:18,500 - lava_callback - ERROR - Error processing callback for node 69bf4e48f1004e82292e3d79: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32898 - "GET / HTTP/1.0" 200 OK 2026-03-24 02:06:10,539 - lava_callback - INFO - Received callback for node 69c1f15a0efaf21d93106a51 INFO: 172.21.0.1:32906 - "POST /node/69c1f15a0efaf21d93106a51 HTTP/1.0" 202 Accepted 2026-03-24 02:06:10,546 - lava_callback - INFO - Starting async job submit for node 69c1f15a0efaf21d93106a51 2026-03-24 02:06:10,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:06:10,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1f15a0efaf21d93106a51 HTTP/1.1" 200 None 2026-03-24 02:06:10,592 - lava_callback - DEBUG - Processing node 69c1f15a0efaf21d93106a51: baseline-arm64 2026-03-24 02:06:10,724 - lava_callback - WARNING - Failed to upload log 2026-03-24 02:06:10,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 02:06:10,994 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 02:06:10,996 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1f15a0efaf21d93106a51/lava-logs.yaml 2026-03-24 02:06:10,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 02:06:11,137 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 02:06:11,139 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1f15a0efaf21d93106a51/lava_callback.json.gz 2026-03-24 02:06:11,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:06:11,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1f15a0efaf21d93106a51 HTTP/1.1" 200 None 2026-03-24 02:06:11,186 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:06:11,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-24 02:06:11,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:06:11,231 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1f15a0efaf21d93106a51 HTTP/1.1" 200 None 2026-03-24 02:06:11,289 - lava_callback - INFO - Completed processing callback for node 69c1f15a0efaf21d93106a51 INFO: 172.21.0.1:32912 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:32928 - "GET / HTTP/1.0" 200 OK 2026-03-24 02:06:18,487 - lava_callback - INFO - Received callback for node 69bf4e95f1004e82292e3d89 INFO: 172.21.0.1:32932 - "POST /node/69bf4e95f1004e82292e3d89 HTTP/1.0" 202 Accepted 2026-03-24 02:06:18,493 - lava_callback - INFO - Starting async job submit for node 69bf4e95f1004e82292e3d89 2026-03-24 02:06:18,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:06:18,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf4e95f1004e82292e3d89 HTTP/1.1" 200 None 2026-03-24 02:06:18,508 - lava_callback - DEBUG - Processing node 69bf4e95f1004e82292e3d89: baseline-arm64-clabbe 2026-03-24 02:06:18,509 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf4e95f1004e82292e3d89 2026-03-24 02:06:18,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:06:18,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf4e95f1004e82292e3d89 HTTP/1.1" 200 None 2026-03-24 02:06:18,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:06:18,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-24 02:06:18,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:06:18,563 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf4e95f1004e82292e3d89 HTTP/1.1" 200 None 2026-03-24 02:06:18,565 - lava_callback - ERROR - Error processing callback for node 69bf4e95f1004e82292e3d89: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56384 - "GET / HTTP/1.0" 200 OK 2026-03-24 02:06:41,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:06:41,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:56264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54984 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54672 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50374 - "GET / HTTP/1.0" 200 OK 2026-03-24 02:08:58,617 - lava_callback - INFO - Received callback for node 69bf4f2cf1004e82292e3da2 INFO: 172.21.0.1:50388 - "POST /node/69bf4f2cf1004e82292e3da2 HTTP/1.0" 202 Accepted 2026-03-24 02:08:58,623 - lava_callback - INFO - Starting async job submit for node 69bf4f2cf1004e82292e3da2 2026-03-24 02:08:58,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:08:58,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf4f2cf1004e82292e3da2 HTTP/1.1" 200 None 2026-03-24 02:08:58,637 - lava_callback - DEBUG - Processing node 69bf4f2cf1004e82292e3da2: baseline-arm64-clabbe 2026-03-24 02:08:58,638 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf4f2cf1004e82292e3da2 2026-03-24 02:08:58,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:08:58,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf4f2cf1004e82292e3da2 HTTP/1.1" 200 None 2026-03-24 02:08:58,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:08:58,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21b717e5cb936602bc2b HTTP/1.1" 200 None 2026-03-24 02:08:58,669 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:08:58,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf4f2cf1004e82292e3da2 HTTP/1.1" 200 None 2026-03-24 02:08:58,691 - lava_callback - ERROR - Error processing callback for node 69bf4f2cf1004e82292e3da2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46294 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43764 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57396 - "GET / HTTP/1.0" 200 OK 2026-03-24 02:10:38,823 - lava_callback - INFO - Received callback for node 69c1f20c0efaf21d93106aa1 INFO: 172.21.0.1:57400 - "POST /node/69c1f20c0efaf21d93106aa1 HTTP/1.0" 202 Accepted 2026-03-24 02:10:38,830 - lava_callback - INFO - Starting async job submit for node 69c1f20c0efaf21d93106aa1 2026-03-24 02:10:38,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:10:38,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1f20c0efaf21d93106aa1 HTTP/1.1" 200 None 2026-03-24 02:10:38,867 - lava_callback - DEBUG - Processing node 69c1f20c0efaf21d93106aa1: baseline-arm64 2026-03-24 02:10:39,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 02:10:39,487 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 02:10:39,489 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1f20c0efaf21d93106aa1/log.txt.gz 2026-03-24 02:10:39,743 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 02:10:39,979 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 02:10:39,981 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1f20c0efaf21d93106aa1/lava-logs.yaml 2026-03-24 02:10:39,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 02:10:40,184 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 02:10:40,186 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1f20c0efaf21d93106aa1/lava_callback.json.gz 2026-03-24 02:10:40,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:10:40,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1f20c0efaf21d93106aa1 HTTP/1.1" 200 None 2026-03-24 02:10:40,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:10:40,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-24 02:10:40,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:10:40,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1f20c0efaf21d93106aa1 HTTP/1.1" 200 None 2026-03-24 02:10:40,338 - lava_callback - INFO - Completed processing callback for node 69c1f20c0efaf21d93106aa1 2026-03-24 02:10:41,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:10:41,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:45528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55838 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44736 - "GET / HTTP/1.0" 200 OK 2026-03-24 02:11:58,257 - lava_callback - INFO - Received callback for node 69c1f2bb0efaf21d93106af4 INFO: 172.21.0.1:44744 - "POST /node/69c1f2bb0efaf21d93106af4 HTTP/1.0" 202 Accepted 2026-03-24 02:11:58,264 - lava_callback - INFO - Starting async job submit for node 69c1f2bb0efaf21d93106af4 2026-03-24 02:11:58,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:11:58,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1f2bb0efaf21d93106af4 HTTP/1.1" 200 None 2026-03-24 02:11:58,310 - lava_callback - DEBUG - Processing node 69c1f2bb0efaf21d93106af4: baseline-arm64 2026-03-24 02:11:58,471 - lava_callback - WARNING - Failed to upload log 2026-03-24 02:11:58,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 02:11:58,764 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 02:11:58,766 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1f2bb0efaf21d93106af4/lava-logs.yaml 2026-03-24 02:11:58,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 02:11:58,908 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 02:11:58,910 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1f2bb0efaf21d93106af4/lava_callback.json.gz 2026-03-24 02:11:58,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:11:58,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1f2bb0efaf21d93106af4 HTTP/1.1" 200 None 2026-03-24 02:11:58,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:11:58,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-24 02:11:58,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:11:59,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1f2bb0efaf21d93106af4 HTTP/1.1" 200 None 2026-03-24 02:11:59,068 - lava_callback - INFO - Completed processing callback for node 69c1f2bb0efaf21d93106af4 INFO: 172.21.0.1:57636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54780 - "GET / HTTP/1.0" 200 OK 2026-03-24 02:12:11,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:12:11,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:54796 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54802 - "GET / HTTP/1.0" 200 OK 2026-03-24 02:12:17,340 - lava_callback - INFO - Received callback for node 69c1ee4f0efaf21d931068f4 INFO: 172.21.0.1:54816 - "POST /node/69c1ee4f0efaf21d931068f4 HTTP/1.0" 202 Accepted 2026-03-24 02:12:17,349 - lava_callback - INFO - Starting async job submit for node 69c1ee4f0efaf21d931068f4 2026-03-24 02:12:17,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:12:17,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1ee4f0efaf21d931068f4 HTTP/1.1" 200 None 2026-03-24 02:12:17,416 - lava_callback - DEBUG - Processing node 69c1ee4f0efaf21d931068f4: baseline-nfs-arm64 2026-03-24 02:12:18,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 02:12:18,718 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 02:12:18,720 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c1ee4f0efaf21d931068f4/log.txt.gz 2026-03-24 02:12:19,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 02:12:19,625 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 02:12:19,628 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c1ee4f0efaf21d931068f4/lava-logs.yaml 2026-03-24 02:12:19,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 02:12:19,836 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 02:12:19,837 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c1ee4f0efaf21d931068f4/lava_callback.json.gz DEBUG: Setup failed for 69c1ee4f0efaf21d931068f4. Transitting job node result: incomplete -> fail 2026-03-24 02:12:19,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:12:19,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1ee4f0efaf21d931068f4 HTTP/1.1" 200 None 2026-03-24 02:12:19,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:12:19,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-24 02:12:19,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:12:19,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1ee4f0efaf21d931068f4 HTTP/1.1" 200 None 2026-03-24 02:12:20,103 - lava_callback - INFO - Completed processing callback for node 69c1ee4f0efaf21d931068f4 INFO: 172.21.0.1:41580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37350 - "GET / HTTP/1.0" 200 OK 2026-03-24 02:12:38,716 - lava_callback - INFO - Received callback for node 69bf500cf1004e82292e3dcd INFO: 172.21.0.1:37352 - "POST /node/69bf500cf1004e82292e3dcd HTTP/1.0" 202 Accepted 2026-03-24 02:12:38,721 - lava_callback - INFO - Starting async job submit for node 69bf500cf1004e82292e3dcd 2026-03-24 02:12:38,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:12:38,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf500cf1004e82292e3dcd HTTP/1.1" 200 None 2026-03-24 02:12:38,746 - lava_callback - DEBUG - Processing node 69bf500cf1004e82292e3dcd: baseline-arm64-clabbe 2026-03-24 02:12:38,747 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf500cf1004e82292e3dcd 2026-03-24 02:12:38,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:12:38,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf500cf1004e82292e3dcd HTTP/1.1" 200 None 2026-03-24 02:12:38,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:12:38,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21b717e5cb936602bc2b HTTP/1.1" 200 None 2026-03-24 02:12:38,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:12:38,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf500cf1004e82292e3dcd HTTP/1.1" 200 None 2026-03-24 02:12:38,798 - lava_callback - ERROR - Error processing callback for node 69bf500cf1004e82292e3dcd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-24 02:12:41,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:12:41,149 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:37174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40660 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33282 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45616 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45632 - "GET / HTTP/1.0" 200 OK 2026-03-24 02:15:18,779 - lava_callback - INFO - Received callback for node 69bf50a1f1004e82292e3de6 INFO: 172.21.0.1:45642 - "POST /node/69bf50a1f1004e82292e3de6 HTTP/1.0" 202 Accepted 2026-03-24 02:15:18,785 - lava_callback - INFO - Starting async job submit for node 69bf50a1f1004e82292e3de6 2026-03-24 02:15:18,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:15:18,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf50a1f1004e82292e3de6 HTTP/1.1" 200 None 2026-03-24 02:15:18,801 - lava_callback - DEBUG - Processing node 69bf50a1f1004e82292e3de6: baseline-arm64-clabbe 2026-03-24 02:15:18,801 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf50a1f1004e82292e3de6 2026-03-24 02:15:18,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:15:18,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf50a1f1004e82292e3de6 HTTP/1.1" 200 None 2026-03-24 02:15:18,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:15:18,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21b717e5cb936602bc2b HTTP/1.1" 200 None 2026-03-24 02:15:18,830 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:15:18,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf50a1f1004e82292e3de6 HTTP/1.1" 200 None 2026-03-24 02:15:18,856 - lava_callback - ERROR - Error processing callback for node 69bf50a1f1004e82292e3de6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59258 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59268 - "GET / HTTP/1.0" 200 OK 2026-03-24 02:16:18,827 - lava_callback - INFO - Received callback for node 69bf50edf1004e82292e3dfe INFO: 172.21.0.1:59284 - "POST /node/69bf50edf1004e82292e3dfe HTTP/1.0" 202 Accepted 2026-03-24 02:16:18,834 - lava_callback - INFO - Starting async job submit for node 69bf50edf1004e82292e3dfe 2026-03-24 02:16:18,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:16:18,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf50edf1004e82292e3dfe HTTP/1.1" 200 None 2026-03-24 02:16:18,850 - lava_callback - DEBUG - Processing node 69bf50edf1004e82292e3dfe: baseline-arm64-clabbe 2026-03-24 02:16:18,850 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf50edf1004e82292e3dfe 2026-03-24 02:16:18,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:16:18,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf50edf1004e82292e3dfe HTTP/1.1" 200 None 2026-03-24 02:16:18,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:16:18,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21b717e5cb936602bc2b HTTP/1.1" 200 None 2026-03-24 02:16:18,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:16:18,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf50edf1004e82292e3dfe HTTP/1.1" 200 None 2026-03-24 02:16:18,902 - lava_callback - ERROR - Error processing callback for node 69bf50edf1004e82292e3dfe: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42192 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55072 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58958 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58102 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58114 - "GET / HTTP/1.0" 200 OK 2026-03-24 02:20:19,025 - lava_callback - INFO - Received callback for node 69bf51d1f1004e82292e3e29 INFO: 172.21.0.1:58128 - "POST /node/69bf51d1f1004e82292e3e29 HTTP/1.0" 202 Accepted 2026-03-24 02:20:19,030 - lava_callback - INFO - Starting async job submit for node 69bf51d1f1004e82292e3e29 2026-03-24 02:20:19,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:20:19,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf51d1f1004e82292e3e29 HTTP/1.1" 200 None 2026-03-24 02:20:19,047 - lava_callback - DEBUG - Processing node 69bf51d1f1004e82292e3e29: baseline-arm64-clabbe 2026-03-24 02:20:19,047 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf51d1f1004e82292e3e29 2026-03-24 02:20:19,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:20:19,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf51d1f1004e82292e3e29 HTTP/1.1" 200 None 2026-03-24 02:20:19,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:20:19,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21b717e5cb936602bc2b HTTP/1.1" 200 None 2026-03-24 02:20:19,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:20:19,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf51d1f1004e82292e3e29 HTTP/1.1" 200 None 2026-03-24 02:20:19,105 - lava_callback - ERROR - Error processing callback for node 69bf51d1f1004e82292e3e29: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42856 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44852 - "GET / HTTP/1.0" 200 OK 2026-03-24 02:21:27,872 - lava_callback - INFO - Received callback for node 69c1f36b0efaf21d93106b4d INFO: 172.21.0.1:44868 - "POST /node/69c1f36b0efaf21d93106b4d HTTP/1.0" 202 Accepted 2026-03-24 02:21:27,879 - lava_callback - INFO - Starting async job submit for node 69c1f36b0efaf21d93106b4d 2026-03-24 02:21:27,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:21:27,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1f36b0efaf21d93106b4d HTTP/1.1" 200 None 2026-03-24 02:21:27,935 - lava_callback - DEBUG - Processing node 69c1f36b0efaf21d93106b4d: baseline-arm64 2026-03-24 02:21:28,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 02:21:28,473 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 02:21:28,474 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1f36b0efaf21d93106b4d/log.txt.gz 2026-03-24 02:21:28,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 02:21:28,918 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 02:21:28,921 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1f36b0efaf21d93106b4d/lava-logs.yaml 2026-03-24 02:21:28,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 02:21:29,136 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 02:21:29,137 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1f36b0efaf21d93106b4d/lava_callback.json.gz DEBUG: Setup failed for 69c1f36b0efaf21d93106b4d. Transitting job node result: incomplete -> fail 2026-03-24 02:21:29,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:21:29,236 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1f36b0efaf21d93106b4d HTTP/1.1" 200 None 2026-03-24 02:21:29,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:21:29,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-24 02:21:29,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:21:29,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1f36b0efaf21d93106b4d HTTP/1.1" 200 None 2026-03-24 02:21:29,373 - lava_callback - INFO - Completed processing callback for node 69c1f36b0efaf21d93106b4d INFO: 172.21.0.1:40592 - "GET / HTTP/1.0" 200 OK 2026-03-24 02:21:39,061 - lava_callback - INFO - Received callback for node 69bf521df1004e82292e3e38 INFO: 172.21.0.1:40608 - "POST /node/69bf521df1004e82292e3e38 HTTP/1.0" 202 Accepted 2026-03-24 02:21:39,066 - lava_callback - INFO - Starting async job submit for node 69bf521df1004e82292e3e38 2026-03-24 02:21:39,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:21:39,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf521df1004e82292e3e38 HTTP/1.1" 200 None 2026-03-24 02:21:39,082 - lava_callback - DEBUG - Processing node 69bf521df1004e82292e3e38: baseline-arm64-clabbe 2026-03-24 02:21:39,082 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf521df1004e82292e3e38 2026-03-24 02:21:39,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:21:39,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf521df1004e82292e3e38 HTTP/1.1" 200 None 2026-03-24 02:21:39,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:21:39,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21b717e5cb936602bc2b HTTP/1.1" 200 None 2026-03-24 02:21:39,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:21:39,146 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf521df1004e82292e3e38 HTTP/1.1" 200 None 2026-03-24 02:21:39,148 - lava_callback - ERROR - Error processing callback for node 69bf521df1004e82292e3e38: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-24 02:21:41,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:21:41,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:38212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38034 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38874 - "GET / HTTP/1.0" 200 OK 2026-03-24 02:22:39,083 - lava_callback - INFO - Received callback for node 69bf5268f1004e82292e3e47 INFO: 172.21.0.1:38884 - "POST /node/69bf5268f1004e82292e3e47 HTTP/1.0" 202 Accepted 2026-03-24 02:22:39,089 - lava_callback - INFO - Starting async job submit for node 69bf5268f1004e82292e3e47 2026-03-24 02:22:39,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:22:39,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf5268f1004e82292e3e47 HTTP/1.1" 200 None 2026-03-24 02:22:39,105 - lava_callback - DEBUG - Processing node 69bf5268f1004e82292e3e47: baseline-arm64-clabbe 2026-03-24 02:22:39,105 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf5268f1004e82292e3e47 2026-03-24 02:22:39,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:22:39,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf5268f1004e82292e3e47 HTTP/1.1" 200 None 2026-03-24 02:22:39,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:22:39,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21b717e5cb936602bc2b HTTP/1.1" 200 None 2026-03-24 02:22:39,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:22:39,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf5268f1004e82292e3e47 HTTP/1.1" 200 None 2026-03-24 02:22:39,169 - lava_callback - ERROR - Error processing callback for node 69bf5268f1004e82292e3e47: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35058 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48402 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52436 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55468 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42440 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45058 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43646 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60088 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55460 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33138 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50978 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59652 - "GET / HTTP/1.0" 200 OK 2026-03-24 02:33:31,598 - lava_callback - INFO - Received callback for node 69c1f5930efaf21d93106c39 INFO: 172.21.0.1:42802 - "POST /node/69c1f5930efaf21d93106c39 HTTP/1.0" 202 Accepted 2026-03-24 02:33:31,605 - lava_callback - INFO - Starting async job submit for node 69c1f5930efaf21d93106c39 2026-03-24 02:33:31,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:33:31,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1f5930efaf21d93106c39 HTTP/1.1" 200 None 2026-03-24 02:33:31,645 - lava_callback - DEBUG - Processing node 69c1f5930efaf21d93106c39: baseline-arm64 2026-03-24 02:33:32,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 02:33:32,347 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 02:33:32,349 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1f5930efaf21d93106c39/log.txt.gz 2026-03-24 02:33:32,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 02:33:32,818 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 02:33:32,820 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1f5930efaf21d93106c39/lava-logs.yaml 2026-03-24 02:33:32,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 02:33:33,017 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 02:33:33,019 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1f5930efaf21d93106c39/lava_callback.json.gz 2026-03-24 02:33:33,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:33:33,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1f5930efaf21d93106c39 HTTP/1.1" 200 None 2026-03-24 02:33:33,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:33:33,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-24 02:33:33,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:33:33,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1f5930efaf21d93106c39 HTTP/1.1" 200 None 2026-03-24 02:33:33,174 - lava_callback - INFO - Completed processing callback for node 69c1f5930efaf21d93106c39 INFO: 172.21.0.1:42808 - "GET / HTTP/1.0" 200 OK 2026-03-24 02:33:41,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:33:41,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:56450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51464 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45182 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33826 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34762 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44980 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35808 - "GET / HTTP/1.0" 200 OK 2026-03-24 02:38:36,172 - lava_callback - INFO - Received callback for node 69c1e3c00efaf21d931060a9 INFO: 172.21.0.1:36742 - "POST /node/69c1e3c00efaf21d931060a9 HTTP/1.0" 202 Accepted 2026-03-24 02:38:36,179 - lava_callback - INFO - Starting async job submit for node 69c1e3c00efaf21d931060a9 2026-03-24 02:38:36,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:38:36,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3c00efaf21d931060a9 HTTP/1.1" 200 None 2026-03-24 02:38:36,217 - lava_callback - DEBUG - Processing node 69c1e3c00efaf21d931060a9: baseline-arm64-broonie 2026-03-24 02:38:36,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:36752 - "GET / HTTP/1.0" 200 OK 2026-03-24 02:38:36,615 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 02:38:36,617 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c1e3c00efaf21d931060a9/log.txt.gz 2026-03-24 02:38:36,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 02:38:36,964 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 02:38:36,966 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c1e3c00efaf21d931060a9/lava-logs.yaml 2026-03-24 02:38:36,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 02:38:37,128 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 02:38:37,129 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c1e3c00efaf21d931060a9/lava_callback.json.gz 2026-03-24 02:38:37,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:38:37,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3c00efaf21d931060a9 HTTP/1.1" 200 None 2026-03-24 02:38:37,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:38:37,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-24 02:38:37,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:38:37,236 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e3c00efaf21d931060a9 HTTP/1.1" 200 None 2026-03-24 02:38:37,242 - lava_callback - INFO - Completed processing callback for node 69c1e3c00efaf21d931060a9 2026-03-24 02:38:41,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:38:41,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-24 02:38:43,303 - lava_callback - INFO - Received callback for node 69c1f5110efaf21d93106c00 INFO: 172.21.0.1:40434 - "POST /node/69c1f5110efaf21d93106c00 HTTP/1.0" 202 Accepted 2026-03-24 02:38:43,310 - lava_callback - INFO - Starting async job submit for node 69c1f5110efaf21d93106c00 2026-03-24 02:38:43,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:38:43,389 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1f5110efaf21d93106c00 HTTP/1.1" 200 None 2026-03-24 02:38:43,390 - lava_callback - DEBUG - Processing node 69c1f5110efaf21d93106c00: baseline-arm 2026-03-24 02:38:43,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 02:38:44,237 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 02:38:44,238 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c1f5110efaf21d93106c00/log.txt.gz 2026-03-24 02:38:44,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 02:38:44,821 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 02:38:44,823 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c1f5110efaf21d93106c00/lava-logs.yaml 2026-03-24 02:38:44,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 02:38:44,993 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 02:38:44,995 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c1f5110efaf21d93106c00/lava_callback.json.gz DEBUG: Setup failed for 69c1f5110efaf21d93106c00. Transitting job node result: incomplete -> fail 2026-03-24 02:38:45,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:38:45,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1f5110efaf21d93106c00 HTTP/1.1" 200 None 2026-03-24 02:38:45,140 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:38:45,157 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-24 02:38:45,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:38:45,183 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1f5110efaf21d93106c00 HTTP/1.1" 200 None 2026-03-24 02:38:45,285 - lava_callback - INFO - Completed processing callback for node 69c1f5110efaf21d93106c00 INFO: 172.21.0.1:40446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51392 - "GET / HTTP/1.0" 200 OK 2026-03-24 02:38:59,505 - lava_callback - INFO - Received callback for node 69c1f6c40efaf21d93106cc2 INFO: 172.21.0.1:51396 - "POST /node/69c1f6c40efaf21d93106cc2 HTTP/1.0" 202 Accepted 2026-03-24 02:38:59,511 - lava_callback - INFO - Starting async job submit for node 69c1f6c40efaf21d93106cc2 2026-03-24 02:38:59,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:38:59,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1f6c40efaf21d93106cc2 HTTP/1.1" 200 None 2026-03-24 02:38:59,552 - lava_callback - DEBUG - Processing node 69c1f6c40efaf21d93106cc2: baseline-nfs-arm64 2026-03-24 02:39:00,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 02:39:00,187 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 02:39:00,188 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c1f6c40efaf21d93106cc2/log.txt.gz 2026-03-24 02:39:00,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 02:39:00,669 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 02:39:00,671 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c1f6c40efaf21d93106cc2/lava-logs.yaml 2026-03-24 02:39:00,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 02:39:00,844 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 02:39:00,846 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c1f6c40efaf21d93106cc2/lava_callback.json.gz 2026-03-24 02:39:00,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:39:00,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1f6c40efaf21d93106cc2 HTTP/1.1" 200 None 2026-03-24 02:39:00,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:39:00,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-24 02:39:00,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:39:00,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1f6c40efaf21d93106cc2 HTTP/1.1" 200 None 2026-03-24 02:39:01,009 - lava_callback - INFO - Completed processing callback for node 69c1f6c40efaf21d93106cc2 INFO: 172.21.0.1:43752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40408 - "GET / HTTP/1.0" 200 OK 2026-03-24 02:39:11,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:39:11,235 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:40410 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33646 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56438 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45206 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58150 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34292 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46404 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42598 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38158 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38068 - "GET / HTTP/1.0" 200 OK 2026-03-24 02:48:14,639 - lava_callback - INFO - Received callback for node 69c1ea040efaf21d93106684 INFO: 172.21.0.1:38072 - "POST /node/69c1ea040efaf21d93106684 HTTP/1.0" 202 Accepted 2026-03-24 02:48:14,645 - lava_callback - INFO - Starting async job submit for node 69c1ea040efaf21d93106684 2026-03-24 02:48:14,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:48:14,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1ea040efaf21d93106684 HTTP/1.1" 200 None 2026-03-24 02:48:14,720 - lava_callback - DEBUG - Processing node 69c1ea040efaf21d93106684: ltp-smoke INFO: 172.21.0.1:38084 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-24 02:48:15,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 02:48:15,339 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 02:48:15,340 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c1ea040efaf21d93106684/log.txt.gz 2026-03-24 02:48:15,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 02:48:15,876 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 02:48:15,878 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c1ea040efaf21d93106684/lava-logs.yaml 2026-03-24 02:48:15,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 02:48:16,067 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 02:48:16,069 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c1ea040efaf21d93106684/lava_callback.json.gz 2026-03-24 02:48:16,186 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:48:16,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1ea040efaf21d93106684 HTTP/1.1" 200 None 2026-03-24 02:48:16,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:48:16,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8340efaf21d93103549 HTTP/1.1" 200 None 2026-03-24 02:48:16,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:48:16,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1ea040efaf21d93106684 HTTP/1.1" 200 None 2026-03-24 02:48:16,390 - lava_callback - INFO - Completed processing callback for node 69c1ea040efaf21d93106684 INFO: 172.21.0.1:38086 - "GET / HTTP/1.0" 200 OK 2026-03-24 02:48:20,439 - lava_callback - INFO - Received callback for node 69bf5863f1004e82292e3f73 INFO: 172.21.0.1:34156 - "POST /node/69bf5863f1004e82292e3f73 HTTP/1.0" 202 Accepted 2026-03-24 02:48:20,444 - lava_callback - INFO - Starting async job submit for node 69bf5863f1004e82292e3f73 2026-03-24 02:48:20,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:48:20,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf5863f1004e82292e3f73 HTTP/1.1" 200 None 2026-03-24 02:48:20,461 - lava_callback - DEBUG - Processing node 69bf5863f1004e82292e3f73: baseline-arm-clabbe 2026-03-24 02:48:20,461 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf5863f1004e82292e3f73 2026-03-24 02:48:20,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:48:20,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf5863f1004e82292e3f73 HTTP/1.1" 200 None 2026-03-24 02:48:20,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:48:20,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-24 02:48:20,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:48:20,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf5863f1004e82292e3f73 HTTP/1.1" 200 None 2026-03-24 02:48:20,532 - lava_callback - ERROR - Error processing callback for node 69bf5863f1004e82292e3f73: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54176 - "GET / HTTP/1.0" 200 OK 2026-03-24 02:48:41,239 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:48:41,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:56284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47112 - "GET / HTTP/1.0" 200 OK 2026-03-24 02:49:00,163 - lava_callback - INFO - Received callback for node 69bf5888f1004e82292e3f7b INFO: 172.21.0.1:41410 - "POST /node/69bf5888f1004e82292e3f7b HTTP/1.0" 202 Accepted 2026-03-24 02:49:00,168 - lava_callback - INFO - Starting async job submit for node 69bf5888f1004e82292e3f7b 2026-03-24 02:49:00,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:49:00,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf5888f1004e82292e3f7b HTTP/1.1" 200 None 2026-03-24 02:49:00,191 - lava_callback - DEBUG - Processing node 69bf5888f1004e82292e3f7b: baseline-arm-clabbe 2026-03-24 02:49:00,192 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf5888f1004e82292e3f7b 2026-03-24 02:49:00,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:49:00,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf5888f1004e82292e3f7b HTTP/1.1" 200 None 2026-03-24 02:49:00,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:49:00,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-24 02:49:00,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:49:00,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf5888f1004e82292e3f7b HTTP/1.1" 200 None 2026-03-24 02:49:00,258 - lava_callback - ERROR - Error processing callback for node 69bf5888f1004e82292e3f7b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48414 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48430 - "GET / HTTP/1.0" 200 OK 2026-03-24 02:49:20,239 - lava_callback - INFO - Received callback for node 69bf58acf1004e82292e3f8c INFO: 172.21.0.1:53210 - "POST /node/69bf58acf1004e82292e3f8c HTTP/1.0" 202 Accepted 2026-03-24 02:49:20,245 - lava_callback - INFO - Starting async job submit for node 69bf58acf1004e82292e3f8c 2026-03-24 02:49:20,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:49:20,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf58acf1004e82292e3f8c HTTP/1.1" 200 None 2026-03-24 02:49:20,261 - lava_callback - DEBUG - Processing node 69bf58acf1004e82292e3f8c: baseline-arm-clabbe 2026-03-24 02:49:20,261 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf58acf1004e82292e3f8c 2026-03-24 02:49:20,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:49:20,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf58acf1004e82292e3f8c HTTP/1.1" 200 None 2026-03-24 02:49:20,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:49:20,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-24 02:49:20,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:49:20,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf58acf1004e82292e3f8c HTTP/1.1" 200 None 2026-03-24 02:49:20,332 - lava_callback - ERROR - Error processing callback for node 69bf58acf1004e82292e3f8c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56774 - "GET / HTTP/1.0" 200 OK 2026-03-24 02:50:00,207 - lava_callback - INFO - Received callback for node 69bf58d0f1004e82292e3fa0 INFO: 172.21.0.1:37284 - "POST /node/69bf58d0f1004e82292e3fa0 HTTP/1.0" 202 Accepted 2026-03-24 02:50:00,212 - lava_callback - INFO - Starting async job submit for node 69bf58d0f1004e82292e3fa0 2026-03-24 02:50:00,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:50:00,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf58d0f1004e82292e3fa0 HTTP/1.1" 200 None 2026-03-24 02:50:00,229 - lava_callback - DEBUG - Processing node 69bf58d0f1004e82292e3fa0: baseline-arm-clabbe 2026-03-24 02:50:00,229 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf58d0f1004e82292e3fa0 2026-03-24 02:50:00,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:50:00,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf58d0f1004e82292e3fa0 HTTP/1.1" 200 None 2026-03-24 02:50:00,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:50:00,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-24 02:50:00,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:50:00,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf58d0f1004e82292e3fa0 HTTP/1.1" 200 None 2026-03-24 02:50:00,301 - lava_callback - ERROR - Error processing callback for node 69bf58d0f1004e82292e3fa0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60780 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56096 - "GET / HTTP/1.0" 200 OK 2026-03-24 02:50:40,401 - lava_callback - INFO - Received callback for node 69bf58f5f1004e82292e3fb2 INFO: 172.21.0.1:58110 - "POST /node/69bf58f5f1004e82292e3fb2 HTTP/1.0" 202 Accepted 2026-03-24 02:50:40,407 - lava_callback - INFO - Starting async job submit for node 69bf58f5f1004e82292e3fb2 2026-03-24 02:50:40,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:50:40,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf58f5f1004e82292e3fb2 HTTP/1.1" 200 None 2026-03-24 02:50:40,423 - lava_callback - DEBUG - Processing node 69bf58f5f1004e82292e3fb2: baseline-arm-clabbe 2026-03-24 02:50:40,423 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf58f5f1004e82292e3fb2 2026-03-24 02:50:40,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:50:40,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf58f5f1004e82292e3fb2 HTTP/1.1" 200 None 2026-03-24 02:50:40,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:50:40,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-24 02:50:40,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:50:40,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf58f5f1004e82292e3fb2 HTTP/1.1" 200 None 2026-03-24 02:50:40,485 - lava_callback - ERROR - Error processing callback for node 69bf58f5f1004e82292e3fb2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49694 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39172 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59690 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49956 - "GET / HTTP/1.0" 200 OK 2026-03-24 02:53:44,107 - lava_callback - INFO - Received callback for node 69c1f69e0efaf21d93106cad INFO: 172.21.0.1:44104 - "POST /node/69c1f69e0efaf21d93106cad HTTP/1.0" 202 Accepted 2026-03-24 02:53:44,153 - lava_callback - INFO - Starting async job submit for node 69c1f69e0efaf21d93106cad 2026-03-24 02:53:44,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:53:44,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1f69e0efaf21d93106cad HTTP/1.1" 200 None 2026-03-24 02:53:44,193 - lava_callback - DEBUG - Processing node 69c1f69e0efaf21d93106cad: baseline-arm-pengutronix 2026-03-24 02:53:44,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 02:53:44,445 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 02:53:44,447 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c1f69e0efaf21d93106cad/log.txt.gz 2026-03-24 02:53:44,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 02:53:44,694 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 02:53:44,696 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c1f69e0efaf21d93106cad/lava-logs.yaml 2026-03-24 02:53:44,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 02:53:44,877 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 02:53:44,879 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c1f69e0efaf21d93106cad/lava_callback.json.gz DEBUG: Setup failed for 69c1f69e0efaf21d93106cad. Transitting job node result: incomplete -> fail 2026-03-24 02:53:44,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:53:44,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1f69e0efaf21d93106cad HTTP/1.1" 200 None 2026-03-24 02:53:44,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:53:44,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-24 02:53:44,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:53:44,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1f69e0efaf21d93106cad HTTP/1.1" 200 None 2026-03-24 02:53:45,052 - lava_callback - INFO - Completed processing callback for node 69c1f69e0efaf21d93106cad INFO: 172.21.0.1:44108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36412 - "GET / HTTP/1.0" 200 OK 2026-03-24 02:54:11,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:54:11,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:36418 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52550 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53156 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48532 - "GET / HTTP/1.0" 200 OK 2026-03-24 02:57:09,063 - lava_callback - INFO - Received callback for node 69c1fbc30efaf21d93106efe INFO: 172.21.0.1:48544 - "POST /node/69c1fbc30efaf21d93106efe HTTP/1.0" 202 Accepted 2026-03-24 02:57:09,070 - lava_callback - INFO - Starting async job submit for node 69c1fbc30efaf21d93106efe 2026-03-24 02:57:09,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:57:09,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1fbc30efaf21d93106efe HTTP/1.1" 200 None 2026-03-24 02:57:09,126 - lava_callback - DEBUG - Processing node 69c1fbc30efaf21d93106efe: baseline-arm64 2026-03-24 02:57:09,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 02:57:09,624 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 02:57:09,626 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1fbc30efaf21d93106efe/log.txt.gz 2026-03-24 02:57:09,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 02:57:10,071 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 02:57:10,073 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1fbc30efaf21d93106efe/lava-logs.yaml 2026-03-24 02:57:10,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 02:57:10,250 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 02:57:10,252 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1fbc30efaf21d93106efe/lava_callback.json.gz DEBUG: Setup failed for 69c1fbc30efaf21d93106efe. Transitting job node result: incomplete -> fail 2026-03-24 02:57:10,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:57:10,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1fbc30efaf21d93106efe HTTP/1.1" 200 None 2026-03-24 02:57:10,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:57:10,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-24 02:57:10,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:57:10,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1fbc30efaf21d93106efe HTTP/1.1" 200 None 2026-03-24 02:57:10,483 - lava_callback - INFO - Completed processing callback for node 69c1fbc30efaf21d93106efe INFO: 172.21.0.1:53300 - "GET / HTTP/1.0" 200 OK 2026-03-24 02:57:11,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:57:11,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:53308 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43640 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.21.0.1:43646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48646 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47504 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60146 - "GET / HTTP/1.0" 200 OK 2026-03-24 02:59:34,079 - lava_callback - INFO - Received callback for node 69c1e3b90efaf21d93106094 INFO: 172.21.0.1:43480 - "POST /node/69c1e3b90efaf21d93106094 HTTP/1.0" 202 Accepted 2026-03-24 02:59:34,086 - lava_callback - INFO - Starting async job submit for node 69c1e3b90efaf21d93106094 2026-03-24 02:59:34,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:59:34,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3b90efaf21d93106094 HTTP/1.1" 200 None 2026-03-24 02:59:34,124 - lava_callback - DEBUG - Processing node 69c1e3b90efaf21d93106094: baseline-arm64-broonie 2026-03-24 02:59:34,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 02:59:34,574 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 02:59:34,576 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c1e3b90efaf21d93106094/log.txt.gz 2026-03-24 02:59:34,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 02:59:34,947 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 02:59:34,949 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c1e3b90efaf21d93106094/lava-logs.yaml 2026-03-24 02:59:34,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 02:59:35,116 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 02:59:35,118 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c1e3b90efaf21d93106094/lava_callback.json.gz 2026-03-24 02:59:35,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:59:35,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3b90efaf21d93106094 HTTP/1.1" 200 None 2026-03-24 02:59:35,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:59:35,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-24 02:59:35,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:59:35,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e3b90efaf21d93106094 HTTP/1.1" 200 None 2026-03-24 02:59:35,226 - lava_callback - INFO - Completed processing callback for node 69c1e3b90efaf21d93106094 INFO: 172.21.0.1:43492 - "GET / HTTP/1.0" 200 OK 2026-03-24 02:59:41,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 02:59:41,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:58228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51604 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54740 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43270 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37824 - "GET / HTTP/1.0" 200 OK 2026-03-24 03:02:39,219 - lava_callback - INFO - Received callback for node 69c1fa260efaf21d93106e44 INFO: 172.21.0.1:37826 - "POST /node/69c1fa260efaf21d93106e44 HTTP/1.0" 202 Accepted 2026-03-24 03:02:39,228 - lava_callback - INFO - Starting async job submit for node 69c1fa260efaf21d93106e44 2026-03-24 03:02:39,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:02:39,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1fa260efaf21d93106e44 HTTP/1.1" 200 None 2026-03-24 03:02:39,290 - lava_callback - DEBUG - Processing node 69c1fa260efaf21d93106e44: baseline-nfs-arm64 2026-03-24 03:02:40,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 03:02:40,611 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 03:02:40,613 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c1fa260efaf21d93106e44/log.txt.gz 2026-03-24 03:02:41,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 03:02:41,522 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 03:02:41,524 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c1fa260efaf21d93106e44/lava-logs.yaml 2026-03-24 03:02:41,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 03:02:41,734 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 03:02:41,736 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c1fa260efaf21d93106e44/lava_callback.json.gz DEBUG: Setup failed for 69c1fa260efaf21d93106e44. Transitting job node result: incomplete -> fail 2026-03-24 03:02:41,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:02:41,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1fa260efaf21d93106e44 HTTP/1.1" 200 None 2026-03-24 03:02:41,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:02:41,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-24 03:02:41,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:02:41,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1fa260efaf21d93106e44 HTTP/1.1" 200 None 2026-03-24 03:02:42,010 - lava_callback - INFO - Completed processing callback for node 69c1fa260efaf21d93106e44 INFO: 172.21.0.1:49046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39440 - "GET / HTTP/1.0" 200 OK 2026-03-24 03:03:11,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:03:11,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:39454 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54904 - "GET / HTTP/1.0" 200 OK 2026-03-24 03:03:52,843 - lava_callback - INFO - Received callback for node 69c1fcbc0efaf21d93106f78 INFO: 172.21.0.1:35878 - "POST /node/69c1fcbc0efaf21d93106f78 HTTP/1.0" 202 Accepted 2026-03-24 03:03:52,850 - lava_callback - INFO - Starting async job submit for node 69c1fcbc0efaf21d93106f78 2026-03-24 03:03:52,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:03:52,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1fcbc0efaf21d93106f78 HTTP/1.1" 200 None 2026-03-24 03:03:52,888 - lava_callback - DEBUG - Processing node 69c1fcbc0efaf21d93106f78: baseline-arm64 2026-03-24 03:03:53,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 03:03:53,570 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 03:03:53,572 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1fcbc0efaf21d93106f78/log.txt.gz 2026-03-24 03:03:53,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 03:03:54,048 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 03:03:54,050 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1fcbc0efaf21d93106f78/lava-logs.yaml 2026-03-24 03:03:54,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 03:03:54,244 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 03:03:54,246 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c1fcbc0efaf21d93106f78/lava_callback.json.gz 2026-03-24 03:03:54,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:03:54,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1fcbc0efaf21d93106f78 HTTP/1.1" 200 None 2026-03-24 03:03:54,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:03:54,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-24 03:03:54,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:03:54,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1fcbc0efaf21d93106f78 HTTP/1.1" 200 None 2026-03-24 03:03:54,395 - lava_callback - INFO - Completed processing callback for node 69c1fcbc0efaf21d93106f78 INFO: 172.21.0.1:35886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53590 - "GET / HTTP/1.0" 200 OK 2026-03-24 03:04:11,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:04:11,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:53594 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40044 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36202 - "GET / HTTP/1.0" 200 OK 2026-03-24 03:06:00,437 - lava_callback - INFO - Received callback for node 69c1fd160efaf21d93106fa4 INFO: 172.21.0.1:40256 - "POST /node/69c1fd160efaf21d93106fa4 HTTP/1.0" 202 Accepted 2026-03-24 03:06:00,443 - lava_callback - INFO - Starting async job submit for node 69c1fd160efaf21d93106fa4 2026-03-24 03:06:00,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:06:00,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1fd160efaf21d93106fa4 HTTP/1.1" 200 None 2026-03-24 03:06:00,499 - lava_callback - DEBUG - Processing node 69c1fd160efaf21d93106fa4: baseline-nfs-arm64 2026-03-24 03:06:01,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 03:06:01,190 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 03:06:01,192 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c1fd160efaf21d93106fa4/log.txt.gz 2026-03-24 03:06:01,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 03:06:01,732 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 03:06:01,734 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c1fd160efaf21d93106fa4/lava-logs.yaml 2026-03-24 03:06:01,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 03:06:01,999 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 03:06:02,000 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c1fd160efaf21d93106fa4/lava_callback.json.gz 2026-03-24 03:06:02,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:06:02,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1fd160efaf21d93106fa4 HTTP/1.1" 200 None 2026-03-24 03:06:02,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:06:02,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-24 03:06:02,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:06:02,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1fd160efaf21d93106fa4 HTTP/1.1" 200 None 2026-03-24 03:06:02,151 - lava_callback - INFO - Completed processing callback for node 69c1fd160efaf21d93106fa4 INFO: 172.21.0.1:40268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39460 - "GET / HTTP/1.0" 200 OK 2026-03-24 03:06:11,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:06:11,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:39466 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54384 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50488 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49208 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35972 - "GET / HTTP/1.0" 200 OK 2026-03-24 03:09:36,568 - lava_callback - INFO - Received callback for node 69c1e3ba0efaf21d93106097 INFO: 172.21.0.1:35978 - "POST /node/69c1e3ba0efaf21d93106097 HTTP/1.0" 202 Accepted 2026-03-24 03:09:36,574 - lava_callback - INFO - Starting async job submit for node 69c1e3ba0efaf21d93106097 2026-03-24 03:09:36,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:09:36,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3ba0efaf21d93106097 HTTP/1.1" 200 None 2026-03-24 03:09:36,612 - lava_callback - DEBUG - Processing node 69c1e3ba0efaf21d93106097: baseline-arm64-broonie 2026-03-24 03:09:36,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 03:09:36,992 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 03:09:36,993 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c1e3ba0efaf21d93106097/log.txt.gz 2026-03-24 03:09:37,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 03:09:37,332 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 03:09:37,333 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c1e3ba0efaf21d93106097/lava-logs.yaml 2026-03-24 03:09:37,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 03:09:37,497 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 03:09:37,498 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c1e3ba0efaf21d93106097/lava_callback.json.gz 2026-03-24 03:09:37,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:09:37,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3ba0efaf21d93106097 HTTP/1.1" 200 None 2026-03-24 03:09:37,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:09:37,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-24 03:09:37,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:09:37,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e3ba0efaf21d93106097 HTTP/1.1" 200 None 2026-03-24 03:09:37,578 - lava_callback - INFO - Completed processing callback for node 69c1e3ba0efaf21d93106097 2026-03-24 03:09:41,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:09:41,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:56456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49812 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60482 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58986 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48298 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40654 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52474 - "GET / HTTP/1.0" 200 OK 2026-03-24 03:14:41,278 - lava_callback - INFO - Received callback for node 69bf5e9df1004e82292e4023 INFO: 172.21.0.1:60320 - "POST /node/69bf5e9df1004e82292e4023 HTTP/1.0" 202 Accepted 2026-03-24 03:14:41,283 - lava_callback - INFO - Starting async job submit for node 69bf5e9df1004e82292e4023 2026-03-24 03:14:41,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:14:41,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf5e9df1004e82292e4023 HTTP/1.1" 200 None 2026-03-24 03:14:41,300 - lava_callback - DEBUG - Processing node 69bf5e9df1004e82292e4023: baseline-arm64-baylibre 2026-03-24 03:14:41,300 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf5e9df1004e82292e4023 2026-03-24 03:14:41,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:14:41,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf5e9df1004e82292e4023 HTTP/1.1" 200 None 2026-03-24 03:14:41,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:14:41,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef90d5731cbd74145fce HTTP/1.1" 200 None 2026-03-24 03:14:41,334 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:14:41,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf5e9df1004e82292e4023 HTTP/1.1" 200 None 2026-03-24 03:14:41,360 - lava_callback - ERROR - Error processing callback for node 69bf5e9df1004e82292e4023: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39400 - "GET / HTTP/1.0" 200 OK 2026-03-24 03:15:14,907 - lava_callback - INFO - Received callback for node 69c1fd9e0efaf21d93106fdc INFO: 172.21.0.1:39402 - "POST /node/69c1fd9e0efaf21d93106fdc HTTP/1.0" 202 Accepted 2026-03-24 03:15:14,914 - lava_callback - INFO - Starting async job submit for node 69c1fd9e0efaf21d93106fdc 2026-03-24 03:15:14,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:15:14,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1fd9e0efaf21d93106fdc HTTP/1.1" 200 None 2026-03-24 03:15:14,990 - lava_callback - DEBUG - Processing node 69c1fd9e0efaf21d93106fdc: ltp-smoke INFO: 172.21.0.1:39408 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-24 03:15:15,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 03:15:15,640 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 03:15:15,641 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c1fd9e0efaf21d93106fdc/log.txt.gz 2026-03-24 03:15:15,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 03:15:16,305 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 03:15:16,307 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c1fd9e0efaf21d93106fdc/lava-logs.yaml 2026-03-24 03:15:16,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 03:15:16,467 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 03:15:16,469 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c1fd9e0efaf21d93106fdc/lava_callback.json.gz INFO: 172.21.0.1:39412 - "GET / HTTP/1.0" 200 OK 2026-03-24 03:15:16,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:15:16,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1fd9e0efaf21d93106fdc HTTP/1.1" 200 None 2026-03-24 03:15:16,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:15:16,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8340efaf21d93103549 HTTP/1.1" 200 None 2026-03-24 03:15:16,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:15:16,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1fd9e0efaf21d93106fdc HTTP/1.1" 200 None 2026-03-24 03:15:16,761 - lava_callback - INFO - Completed processing callback for node 69c1fd9e0efaf21d93106fdc INFO: 172.21.0.1:54026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33302 - "GET / HTTP/1.0" 200 OK 2026-03-24 03:15:41,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:15:41,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:35398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45438 - "GET / HTTP/1.0" 200 OK 2026-03-24 03:16:01,853 - lava_callback - INFO - Received callback for node 69bf5eedf1004e82292e4033 INFO: 172.21.0.1:36438 - "POST /node/69bf5eedf1004e82292e4033 HTTP/1.0" 202 Accepted 2026-03-24 03:16:01,859 - lava_callback - INFO - Starting async job submit for node 69bf5eedf1004e82292e4033 2026-03-24 03:16:01,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:16:01,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf5eedf1004e82292e4033 HTTP/1.1" 200 None 2026-03-24 03:16:01,877 - lava_callback - DEBUG - Processing node 69bf5eedf1004e82292e4033: baseline-arm64-baylibre 2026-03-24 03:16:01,877 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf5eedf1004e82292e4033 2026-03-24 03:16:01,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:16:01,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf5eedf1004e82292e4033 HTTP/1.1" 200 None 2026-03-24 03:16:01,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:16:01,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef90d5731cbd74145fce HTTP/1.1" 200 None 2026-03-24 03:16:01,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:16:01,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf5eedf1004e82292e4033 HTTP/1.1" 200 None 2026-03-24 03:16:01,934 - lava_callback - ERROR - Error processing callback for node 69bf5eedf1004e82292e4033: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49936 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51442 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51444 - "GET / HTTP/1.0" 200 OK 2026-03-24 03:17:21,182 - lava_callback - INFO - Received callback for node 69bf5f3df1004e82292e4042 INFO: 172.21.0.1:44402 - "POST /node/69bf5f3df1004e82292e4042 HTTP/1.0" 202 Accepted 2026-03-24 03:17:21,187 - lava_callback - INFO - Starting async job submit for node 69bf5f3df1004e82292e4042 2026-03-24 03:17:21,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:17:21,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf5f3df1004e82292e4042 HTTP/1.1" 200 None 2026-03-24 03:17:21,203 - lava_callback - DEBUG - Processing node 69bf5f3df1004e82292e4042: baseline-arm64-baylibre 2026-03-24 03:17:21,203 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf5f3df1004e82292e4042 2026-03-24 03:17:21,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:17:21,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf5f3df1004e82292e4042 HTTP/1.1" 200 None 2026-03-24 03:17:21,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:17:21,231 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef90d5731cbd74145fce HTTP/1.1" 200 None 2026-03-24 03:17:21,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:17:21,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf5f3df1004e82292e4042 HTTP/1.1" 200 None 2026-03-24 03:17:21,261 - lava_callback - ERROR - Error processing callback for node 69bf5f3df1004e82292e4042: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47414 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55644 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35154 - "GET / HTTP/1.0" 200 OK 2026-03-24 03:19:41,321 - lava_callback - INFO - Received callback for node 69bf5fcbf1004e82292e4069 INFO: 172.21.0.1:41716 - "POST /node/69bf5fcbf1004e82292e4069 HTTP/1.0" 202 Accepted 2026-03-24 03:19:41,327 - lava_callback - INFO - Starting async job submit for node 69bf5fcbf1004e82292e4069 2026-03-24 03:19:41,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:19:41,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf5fcbf1004e82292e4069 HTTP/1.1" 200 None 2026-03-24 03:19:41,343 - lava_callback - DEBUG - Processing node 69bf5fcbf1004e82292e4069: baseline-arm64-baylibre 2026-03-24 03:19:41,343 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf5fcbf1004e82292e4069 2026-03-24 03:19:41,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:19:41,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf5fcbf1004e82292e4069 HTTP/1.1" 200 None 2026-03-24 03:19:41,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:19:41,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef90d5731cbd74145fce HTTP/1.1" 200 None 2026-03-24 03:19:41,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:19:41,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf5fcbf1004e82292e4069 HTTP/1.1" 200 None 2026-03-24 03:19:41,416 - lava_callback - ERROR - Error processing callback for node 69bf5fcbf1004e82292e4069: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46194 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42868 - "GET / HTTP/1.0" 200 OK 2026-03-24 03:21:01,536 - lava_callback - INFO - Received callback for node 69bf6017f1004e82292e4075 INFO: 172.21.0.1:47196 - "POST /node/69bf6017f1004e82292e4075 HTTP/1.0" 202 Accepted 2026-03-24 03:21:01,542 - lava_callback - INFO - Starting async job submit for node 69bf6017f1004e82292e4075 2026-03-24 03:21:01,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:21:01,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf6017f1004e82292e4075 HTTP/1.1" 200 None 2026-03-24 03:21:01,558 - lava_callback - DEBUG - Processing node 69bf6017f1004e82292e4075: baseline-arm64-baylibre 2026-03-24 03:21:01,558 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf6017f1004e82292e4075 2026-03-24 03:21:01,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:21:01,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf6017f1004e82292e4075 HTTP/1.1" 200 None 2026-03-24 03:21:01,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:21:01,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef90d5731cbd74145fce HTTP/1.1" 200 None 2026-03-24 03:21:01,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:21:01,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf6017f1004e82292e4075 HTTP/1.1" 200 None 2026-03-24 03:21:01,639 - lava_callback - ERROR - Error processing callback for node 69bf6017f1004e82292e4075: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57340 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58728 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58744 - "GET / HTTP/1.0" 200 OK 2026-03-24 03:22:21,505 - lava_callback - INFO - Received callback for node 69bf6066f1004e82292e4086 INFO: 172.21.0.1:59186 - "POST /node/69bf6066f1004e82292e4086 HTTP/1.0" 202 Accepted 2026-03-24 03:22:21,511 - lava_callback - INFO - Starting async job submit for node 69bf6066f1004e82292e4086 2026-03-24 03:22:21,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:22:21,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf6066f1004e82292e4086 HTTP/1.1" 200 None 2026-03-24 03:22:21,531 - lava_callback - DEBUG - Processing node 69bf6066f1004e82292e4086: baseline-arm64-baylibre 2026-03-24 03:22:21,531 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf6066f1004e82292e4086 2026-03-24 03:22:21,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:22:21,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf6066f1004e82292e4086 HTTP/1.1" 200 None 2026-03-24 03:22:21,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:22:21,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef90d5731cbd74145fce HTTP/1.1" 200 None 2026-03-24 03:22:21,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:22:21,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf6066f1004e82292e4086 HTTP/1.1" 200 None 2026-03-24 03:22:21,584 - lava_callback - ERROR - Error processing callback for node 69bf6066f1004e82292e4086: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56332 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42876 - "GET / HTTP/1.0" 200 OK 2026-03-24 03:23:41,587 - lava_callback - INFO - Received callback for node 69bf60b7f1004e82292e4098 INFO: 172.21.0.1:45120 - "POST /node/69bf60b7f1004e82292e4098 HTTP/1.0" 202 Accepted 2026-03-24 03:23:41,593 - lava_callback - INFO - Starting async job submit for node 69bf60b7f1004e82292e4098 2026-03-24 03:23:41,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:23:41,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf60b7f1004e82292e4098 HTTP/1.1" 200 None 2026-03-24 03:23:41,609 - lava_callback - DEBUG - Processing node 69bf60b7f1004e82292e4098: baseline-arm64-clabbe 2026-03-24 03:23:41,609 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf60b7f1004e82292e4098 2026-03-24 03:23:41,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:23:41,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf60b7f1004e82292e4098 HTTP/1.1" 200 None 2026-03-24 03:23:41,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:23:41,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef90d5731cbd74145fce HTTP/1.1" 200 None 2026-03-24 03:23:41,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:23:41,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf60b7f1004e82292e4098 HTTP/1.1" 200 None 2026-03-24 03:23:41,680 - lava_callback - ERROR - Error processing callback for node 69bf60b7f1004e82292e4098: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45124 - "GET / HTTP/1.0" 200 OK 2026-03-24 03:23:47,545 - lava_callback - INFO - Received callback for node 69c1ff0b0efaf21d93107073 INFO: 172.21.0.1:45138 - "POST /node/69c1ff0b0efaf21d93107073 HTTP/1.0" 202 Accepted 2026-03-24 03:23:47,554 - lava_callback - INFO - Starting async job submit for node 69c1ff0b0efaf21d93107073 2026-03-24 03:23:47,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:23:47,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1ff0b0efaf21d93107073 HTTP/1.1" 200 None 2026-03-24 03:23:47,632 - lava_callback - DEBUG - Processing node 69c1ff0b0efaf21d93107073: baseline-nfs-arm64 2026-03-24 03:23:48,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 03:23:48,957 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 03:23:48,959 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c1ff0b0efaf21d93107073/log.txt.gz 2026-03-24 03:23:49,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 03:23:49,816 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 03:23:49,819 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c1ff0b0efaf21d93107073/lava-logs.yaml 2026-03-24 03:23:49,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 03:23:50,032 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 03:23:50,034 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c1ff0b0efaf21d93107073/lava_callback.json.gz DEBUG: Setup failed for 69c1ff0b0efaf21d93107073. Transitting job node result: incomplete -> fail 2026-03-24 03:23:50,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:23:50,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1ff0b0efaf21d93107073 HTTP/1.1" 200 None 2026-03-24 03:23:50,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:23:50,183 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-24 03:23:50,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:23:50,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1ff0b0efaf21d93107073 HTTP/1.1" 200 None 2026-03-24 03:23:50,341 - lava_callback - INFO - Completed processing callback for node 69c1ff0b0efaf21d93107073 INFO: 172.21.0.1:49886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46298 - "GET / HTTP/1.0" 200 OK 2026-03-24 03:24:11,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:24:11,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:46306 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50844 - "GET / HTTP/1.0" 200 OK 2026-03-24 03:25:01,586 - lava_callback - INFO - Received callback for node 69bf6101f1004e82292e40a4 INFO: 172.21.0.1:49584 - "POST /node/69bf6101f1004e82292e40a4 HTTP/1.0" 202 Accepted 2026-03-24 03:25:01,592 - lava_callback - INFO - Starting async job submit for node 69bf6101f1004e82292e40a4 2026-03-24 03:25:01,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:25:01,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf6101f1004e82292e40a4 HTTP/1.1" 200 None 2026-03-24 03:25:01,607 - lava_callback - DEBUG - Processing node 69bf6101f1004e82292e40a4: baseline-arm64-clabbe 2026-03-24 03:25:01,608 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf6101f1004e82292e40a4 2026-03-24 03:25:01,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:25:01,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf6101f1004e82292e40a4 HTTP/1.1" 200 None 2026-03-24 03:25:01,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:25:01,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef90d5731cbd74145fce HTTP/1.1" 200 None 2026-03-24 03:25:01,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:25:01,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf6101f1004e82292e40a4 HTTP/1.1" 200 None 2026-03-24 03:25:01,662 - lava_callback - ERROR - Error processing callback for node 69bf6101f1004e82292e40a4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47894 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55446 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55460 - "GET / HTTP/1.0" 200 OK 2026-03-24 03:26:21,642 - lava_callback - INFO - Received callback for node 69bf614bf1004e82292e40b6 INFO: 172.21.0.1:46916 - "POST /node/69bf614bf1004e82292e40b6 HTTP/1.0" 202 Accepted 2026-03-24 03:26:21,648 - lava_callback - INFO - Starting async job submit for node 69bf614bf1004e82292e40b6 2026-03-24 03:26:21,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:26:21,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf614bf1004e82292e40b6 HTTP/1.1" 200 None 2026-03-24 03:26:21,663 - lava_callback - DEBUG - Processing node 69bf614bf1004e82292e40b6: baseline-arm64-clabbe 2026-03-24 03:26:21,663 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf614bf1004e82292e40b6 2026-03-24 03:26:21,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:26:21,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf614bf1004e82292e40b6 HTTP/1.1" 200 None 2026-03-24 03:26:21,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:26:21,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef90d5731cbd74145fce HTTP/1.1" 200 None 2026-03-24 03:26:21,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:26:21,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf614bf1004e82292e40b6 HTTP/1.1" 200 None 2026-03-24 03:26:21,715 - lava_callback - ERROR - Error processing callback for node 69bf614bf1004e82292e40b6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46090 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46106 - "GET / HTTP/1.0" 200 OK 2026-03-24 03:27:21,697 - lava_callback - INFO - Received callback for node 69bf6196f1004e82292e40c1 INFO: 172.21.0.1:45906 - "POST /node/69bf6196f1004e82292e40c1 HTTP/1.0" 202 Accepted 2026-03-24 03:27:21,702 - lava_callback - INFO - Starting async job submit for node 69bf6196f1004e82292e40c1 2026-03-24 03:27:21,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:27:21,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf6196f1004e82292e40c1 HTTP/1.1" 200 None 2026-03-24 03:27:21,718 - lava_callback - DEBUG - Processing node 69bf6196f1004e82292e40c1: baseline-arm64-clabbe 2026-03-24 03:27:21,718 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf6196f1004e82292e40c1 2026-03-24 03:27:21,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:27:21,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf6196f1004e82292e40c1 HTTP/1.1" 200 None 2026-03-24 03:27:21,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:27:21,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef90d5731cbd74145fce HTTP/1.1" 200 None 2026-03-24 03:27:21,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:27:21,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf6196f1004e82292e40c1 HTTP/1.1" 200 None 2026-03-24 03:27:21,769 - lava_callback - ERROR - Error processing callback for node 69bf6196f1004e82292e40c1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34596 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49346 - "GET / HTTP/1.0" 200 OK 2026-03-24 03:28:41,739 - lava_callback - INFO - Received callback for node 69bf61e0f1004e82292e40cf INFO: 172.21.0.1:35568 - "POST /node/69bf61e0f1004e82292e40cf HTTP/1.0" 202 Accepted 2026-03-24 03:28:41,745 - lava_callback - INFO - Starting async job submit for node 69bf61e0f1004e82292e40cf 2026-03-24 03:28:41,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:28:41,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf61e0f1004e82292e40cf HTTP/1.1" 200 None 2026-03-24 03:28:41,762 - lava_callback - DEBUG - Processing node 69bf61e0f1004e82292e40cf: baseline-arm64-clabbe 2026-03-24 03:28:41,762 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf61e0f1004e82292e40cf 2026-03-24 03:28:41,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:28:41,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf61e0f1004e82292e40cf HTTP/1.1" 200 None 2026-03-24 03:28:41,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:28:41,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef90d5731cbd74145fce HTTP/1.1" 200 None 2026-03-24 03:28:41,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:28:41,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf61e0f1004e82292e40cf HTTP/1.1" 200 None 2026-03-24 03:28:41,819 - lava_callback - ERROR - Error processing callback for node 69bf61e0f1004e82292e40cf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33978 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45284 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52356 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52362 - "GET / HTTP/1.0" 200 OK 2026-03-24 03:31:21,854 - lava_callback - INFO - Received callback for node 69bf6275f1004e82292e40e0 INFO: 172.21.0.1:56842 - "POST /node/69bf6275f1004e82292e40e0 HTTP/1.0" 202 Accepted 2026-03-24 03:31:21,860 - lava_callback - INFO - Starting async job submit for node 69bf6275f1004e82292e40e0 2026-03-24 03:31:21,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:31:21,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf6275f1004e82292e40e0 HTTP/1.1" 200 None 2026-03-24 03:31:21,876 - lava_callback - DEBUG - Processing node 69bf6275f1004e82292e40e0: baseline-arm64-clabbe 2026-03-24 03:31:21,876 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf6275f1004e82292e40e0 2026-03-24 03:31:21,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:31:21,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf6275f1004e82292e40e0 HTTP/1.1" 200 None 2026-03-24 03:31:21,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:31:21,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef90d5731cbd74145fce HTTP/1.1" 200 None 2026-03-24 03:31:21,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:31:21,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf6275f1004e82292e40e0 HTTP/1.1" 200 None 2026-03-24 03:31:21,943 - lava_callback - ERROR - Error processing callback for node 69bf6275f1004e82292e40e0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45132 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37694 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53580 - "GET / HTTP/1.0" 200 OK 2026-03-24 03:33:27,350 - lava_callback - INFO - Received callback for node 69c1e3b70efaf21d93106091 INFO: 172.21.0.1:53596 - "POST /node/69c1e3b70efaf21d93106091 HTTP/1.0" 202 Accepted 2026-03-24 03:33:27,356 - lava_callback - INFO - Starting async job submit for node 69c1e3b70efaf21d93106091 2026-03-24 03:33:27,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:33:27,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3b70efaf21d93106091 HTTP/1.1" 200 None 2026-03-24 03:33:27,394 - lava_callback - DEBUG - Processing node 69c1e3b70efaf21d93106091: baseline-arm64-broonie 2026-03-24 03:33:27,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 03:33:27,813 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 03:33:27,814 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c1e3b70efaf21d93106091/log.txt.gz 2026-03-24 03:33:27,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 03:33:28,205 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 03:33:28,207 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c1e3b70efaf21d93106091/lava-logs.yaml 2026-03-24 03:33:28,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 03:33:28,361 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 03:33:28,362 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c1e3b70efaf21d93106091/lava_callback.json.gz 2026-03-24 03:33:28,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:33:28,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3b70efaf21d93106091 HTTP/1.1" 200 None 2026-03-24 03:33:28,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:33:28,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-24 03:33:28,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:33:28,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e3b70efaf21d93106091 HTTP/1.1" 200 None 2026-03-24 03:33:28,449 - lava_callback - INFO - Completed processing callback for node 69c1e3b70efaf21d93106091 INFO: 172.21.0.1:58106 - "GET / HTTP/1.0" 200 OK 2026-03-24 03:33:41,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:33:41,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:53682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56784 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35274 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55554 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58116 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49546 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51024 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43216 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54728 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39600 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40126 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45614 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51438 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42846 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52514 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39046 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35382 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39338 - "GET / HTTP/1.0" 200 OK 2026-03-24 03:50:08,019 - lava_callback - INFO - Received callback for node 69c1e6350efaf21d931063db INFO: 172.21.0.1:39342 - "POST /node/69c1e6350efaf21d931063db HTTP/1.0" 202 Accepted 2026-03-24 03:50:08,026 - lava_callback - INFO - Starting async job submit for node 69c1e6350efaf21d931063db 2026-03-24 03:50:08,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:50:08,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e6350efaf21d931063db HTTP/1.1" 200 None 2026-03-24 03:50:08,066 - lava_callback - DEBUG - Processing node 69c1e6350efaf21d931063db: ltp-smoke 2026-03-24 03:50:08,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 03:50:08,576 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 03:50:08,578 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c1e6350efaf21d931063db/log.txt.gz 2026-03-24 03:50:08,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 03:50:09,081 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 03:50:09,083 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c1e6350efaf21d931063db/lava-logs.yaml 2026-03-24 03:50:09,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 03:50:09,265 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 03:50:09,266 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c1e6350efaf21d931063db/lava_callback.json.gz 2026-03-24 03:50:09,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:50:09,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e6350efaf21d931063db HTTP/1.1" 200 None 2026-03-24 03:50:09,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:50:09,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-24 03:50:09,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:50:09,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e6350efaf21d931063db HTTP/1.1" 200 None 2026-03-24 03:50:09,349 - lava_callback - INFO - Completed processing callback for node 69c1e6350efaf21d931063db 2026-03-24 03:50:11,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:50:11,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:59988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59996 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55596 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45566 - "GET / HTTP/1.0" 200 OK 2026-03-24 03:51:45,683 - lava_callback - INFO - Received callback for node 69c1fdab0efaf21d93106fe0 INFO: 172.21.0.1:37026 - "POST /node/69c1fdab0efaf21d93106fe0 HTTP/1.0" 202 Accepted 2026-03-24 03:51:45,729 - lava_callback - INFO - Starting async job submit for node 69c1fdab0efaf21d93106fe0 2026-03-24 03:51:45,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:51:45,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1fdab0efaf21d93106fe0 HTTP/1.1" 200 None 2026-03-24 03:51:45,769 - lava_callback - DEBUG - Processing node 69c1fdab0efaf21d93106fe0: baseline-arm-pengutronix 2026-03-24 03:51:45,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 03:51:46,014 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 03:51:46,016 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c1fdab0efaf21d93106fe0/log.txt.gz 2026-03-24 03:51:46,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 03:51:46,268 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 03:51:46,269 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c1fdab0efaf21d93106fe0/lava-logs.yaml 2026-03-24 03:51:46,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 03:51:46,404 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 03:51:46,405 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c1fdab0efaf21d93106fe0/lava_callback.json.gz DEBUG: Setup failed for 69c1fdab0efaf21d93106fe0. Transitting job node result: incomplete -> fail 2026-03-24 03:51:46,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:51:46,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1fdab0efaf21d93106fe0 HTTP/1.1" 200 None 2026-03-24 03:51:46,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:51:46,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-24 03:51:46,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:37036 - "GET / HTTP/1.0" 200 OK 2026-03-24 03:51:46,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1fdab0efaf21d93106fe0 HTTP/1.1" 200 None 2026-03-24 03:51:46,610 - lava_callback - INFO - Completed processing callback for node 69c1fdab0efaf21d93106fe0 INFO: 172.21.0.1:36416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36152 - "GET / HTTP/1.0" 200 OK 2026-03-24 03:52:11,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:52:11,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:47530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47546 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32994 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60766 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51276 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55550 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38002 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47970 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48204 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48206 - "GET / HTTP/1.0" 200 OK 2026-03-24 03:59:23,970 - lava_callback - INFO - Received callback for node 69bf690df1004e82292e4161 INFO: 172.21.0.1:56706 - "POST /node/69bf690df1004e82292e4161 HTTP/1.0" 202 Accepted 2026-03-24 03:59:23,975 - lava_callback - INFO - Starting async job submit for node 69bf690df1004e82292e4161 2026-03-24 03:59:23,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:59:23,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf690df1004e82292e4161 HTTP/1.1" 200 None 2026-03-24 03:59:23,992 - lava_callback - DEBUG - Processing node 69bf690df1004e82292e4161: baseline-arm64-baylibre 2026-03-24 03:59:23,992 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf690df1004e82292e4161 2026-03-24 03:59:23,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:59:24,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf690df1004e82292e4161 HTTP/1.1" 200 None 2026-03-24 03:59:24,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:59:24,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-24 03:59:24,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 03:59:24,044 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf690df1004e82292e4161 HTTP/1.1" 200 None 2026-03-24 03:59:24,045 - lava_callback - ERROR - Error processing callback for node 69bf690df1004e82292e4161: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50840 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49928 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34316 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34328 - "GET / HTTP/1.0" 200 OK 2026-03-24 04:02:23,722 - lava_callback - INFO - Received callback for node 69bf69c0f1004e82292e417f INFO: 172.21.0.1:57274 - "POST /node/69bf69c0f1004e82292e417f HTTP/1.0" 202 Accepted 2026-03-24 04:02:23,728 - lava_callback - INFO - Starting async job submit for node 69bf69c0f1004e82292e417f 2026-03-24 04:02:23,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 04:02:23,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf69c0f1004e82292e417f HTTP/1.1" 200 None 2026-03-24 04:02:23,744 - lava_callback - DEBUG - Processing node 69bf69c0f1004e82292e417f: baseline-arm64-baylibre 2026-03-24 04:02:23,744 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf69c0f1004e82292e417f 2026-03-24 04:02:23,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 04:02:23,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf69c0f1004e82292e417f HTTP/1.1" 200 None 2026-03-24 04:02:23,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 04:02:23,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-24 04:02:23,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 04:02:23,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf69c0f1004e82292e417f HTTP/1.1" 200 None 2026-03-24 04:02:23,799 - lava_callback - ERROR - Error processing callback for node 69bf69c0f1004e82292e417f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42678 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54324 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59452 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53364 - "GET / HTTP/1.0" 200 OK 2026-03-24 04:05:44,164 - lava_callback - INFO - Received callback for node 69bf6a84f1004e82292e4199 INFO: 172.21.0.1:44418 - "POST /node/69bf6a84f1004e82292e4199 HTTP/1.0" 202 Accepted 2026-03-24 04:05:44,170 - lava_callback - INFO - Starting async job submit for node 69bf6a84f1004e82292e4199 2026-03-24 04:05:44,171 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 04:05:44,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf6a84f1004e82292e4199 HTTP/1.1" 200 None 2026-03-24 04:05:44,212 - lava_callback - DEBUG - Processing node 69bf6a84f1004e82292e4199: baseline-arm64-baylibre 2026-03-24 04:05:44,213 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf6a84f1004e82292e4199 2026-03-24 04:05:44,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 04:05:44,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf6a84f1004e82292e4199 HTTP/1.1" 200 None 2026-03-24 04:05:44,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 04:05:44,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-24 04:05:44,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 04:05:44,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf6a84f1004e82292e4199 HTTP/1.1" 200 None 2026-03-24 04:05:44,303 - lava_callback - ERROR - Error processing callback for node 69bf6a84f1004e82292e4199: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54662 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55908 - "GET / HTTP/1.0" 200 OK 2026-03-24 04:07:03,426 - lava_callback - INFO - Received callback for node 69bf6ad4f1004e82292e41ac INFO: 172.21.0.1:46750 - "POST /node/69bf6ad4f1004e82292e41ac HTTP/1.0" 202 Accepted 2026-03-24 04:07:03,432 - lava_callback - INFO - Starting async job submit for node 69bf6ad4f1004e82292e41ac 2026-03-24 04:07:03,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 04:07:03,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf6ad4f1004e82292e41ac HTTP/1.1" 200 None 2026-03-24 04:07:03,470 - lava_callback - DEBUG - Processing node 69bf6ad4f1004e82292e41ac: baseline-arm64-clabbe 2026-03-24 04:07:03,470 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf6ad4f1004e82292e41ac 2026-03-24 04:07:03,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 04:07:03,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf6ad4f1004e82292e41ac HTTP/1.1" 200 None 2026-03-24 04:07:03,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 04:07:03,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-24 04:07:03,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 04:07:03,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf6ad4f1004e82292e41ac HTTP/1.1" 200 None 2026-03-24 04:07:03,550 - lava_callback - ERROR - Error processing callback for node 69bf6ad4f1004e82292e41ac: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48992 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49890 - "GET / HTTP/1.0" 200 OK 2026-03-24 04:08:03,465 - lava_callback - INFO - Received callback for node 69bf6b1ff1004e82292e41bb INFO: 172.21.0.1:58488 - "POST /node/69bf6b1ff1004e82292e41bb HTTP/1.0" 202 Accepted 2026-03-24 04:08:03,471 - lava_callback - INFO - Starting async job submit for node 69bf6b1ff1004e82292e41bb 2026-03-24 04:08:03,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 04:08:03,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf6b1ff1004e82292e41bb HTTP/1.1" 200 None 2026-03-24 04:08:03,509 - lava_callback - DEBUG - Processing node 69bf6b1ff1004e82292e41bb: baseline-arm64-clabbe 2026-03-24 04:08:03,510 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf6b1ff1004e82292e41bb 2026-03-24 04:08:03,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 04:08:03,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf6b1ff1004e82292e41bb HTTP/1.1" 200 None 2026-03-24 04:08:03,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 04:08:03,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-24 04:08:03,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 04:08:03,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf6b1ff1004e82292e41bb HTTP/1.1" 200 None 2026-03-24 04:08:03,600 - lava_callback - ERROR - Error processing callback for node 69bf6b1ff1004e82292e41bb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36728 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47078 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47082 - "GET / HTTP/1.0" 200 OK 2026-03-24 04:09:23,534 - lava_callback - INFO - Received callback for node 69bf6b6bf1004e82292e41d0 INFO: 172.21.0.1:57640 - "POST /node/69bf6b6bf1004e82292e41d0 HTTP/1.0" 202 Accepted 2026-03-24 04:09:23,540 - lava_callback - INFO - Starting async job submit for node 69bf6b6bf1004e82292e41d0 2026-03-24 04:09:23,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 04:09:23,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf6b6bf1004e82292e41d0 HTTP/1.1" 200 None 2026-03-24 04:09:23,567 - lava_callback - DEBUG - Processing node 69bf6b6bf1004e82292e41d0: baseline-arm64-clabbe 2026-03-24 04:09:23,567 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf6b6bf1004e82292e41d0 2026-03-24 04:09:23,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 04:09:23,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf6b6bf1004e82292e41d0 HTTP/1.1" 200 None 2026-03-24 04:09:23,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 04:09:23,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-24 04:09:23,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 04:09:23,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf6b6bf1004e82292e41d0 HTTP/1.1" 200 None 2026-03-24 04:09:23,665 - lava_callback - ERROR - Error processing callback for node 69bf6b6bf1004e82292e41d0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52938 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38130 - "GET / HTTP/1.0" 200 OK 2026-03-24 04:10:43,475 - lava_callback - INFO - Received callback for node 69bf6bb8f1004e82292e41df INFO: 172.21.0.1:33692 - "POST /node/69bf6bb8f1004e82292e41df HTTP/1.0" 202 Accepted 2026-03-24 04:10:43,480 - lava_callback - INFO - Starting async job submit for node 69bf6bb8f1004e82292e41df 2026-03-24 04:10:43,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 04:10:43,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf6bb8f1004e82292e41df HTTP/1.1" 200 None 2026-03-24 04:10:43,519 - lava_callback - DEBUG - Processing node 69bf6bb8f1004e82292e41df: baseline-arm64-clabbe 2026-03-24 04:10:43,520 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf6bb8f1004e82292e41df 2026-03-24 04:10:43,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 04:10:43,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf6bb8f1004e82292e41df HTTP/1.1" 200 None 2026-03-24 04:10:43,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 04:10:43,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-24 04:10:43,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 04:10:43,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf6bb8f1004e82292e41df HTTP/1.1" 200 None 2026-03-24 04:10:43,589 - lava_callback - ERROR - Error processing callback for node 69bf6bb8f1004e82292e41df: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49034 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57840 - "GET / HTTP/1.0" 200 OK 2026-03-24 04:12:03,553 - lava_callback - INFO - Received callback for node 69bf6c04f1004e82292e41f3 INFO: 172.21.0.1:51878 - "POST /node/69bf6c04f1004e82292e41f3 HTTP/1.0" 202 Accepted 2026-03-24 04:12:03,559 - lava_callback - INFO - Starting async job submit for node 69bf6c04f1004e82292e41f3 2026-03-24 04:12:03,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 04:12:03,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf6c04f1004e82292e41f3 HTTP/1.1" 200 None 2026-03-24 04:12:03,575 - lava_callback - DEBUG - Processing node 69bf6c04f1004e82292e41f3: baseline-arm64-clabbe 2026-03-24 04:12:03,575 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf6c04f1004e82292e41f3 2026-03-24 04:12:03,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 04:12:03,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf6c04f1004e82292e41f3 HTTP/1.1" 200 None 2026-03-24 04:12:03,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 04:12:03,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-24 04:12:03,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 04:12:03,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf6c04f1004e82292e41f3 HTTP/1.1" 200 None 2026-03-24 04:12:03,633 - lava_callback - ERROR - Error processing callback for node 69bf6c04f1004e82292e41f3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55934 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37004 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37016 - "GET / HTTP/1.0" 200 OK 2026-03-24 04:13:23,645 - lava_callback - INFO - Received callback for node 69bf6c51f1004e82292e420c INFO: 172.21.0.1:60080 - "POST /node/69bf6c51f1004e82292e420c HTTP/1.0" 202 Accepted 2026-03-24 04:13:23,651 - lava_callback - INFO - Starting async job submit for node 69bf6c51f1004e82292e420c 2026-03-24 04:13:23,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 04:13:23,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf6c51f1004e82292e420c HTTP/1.1" 200 None 2026-03-24 04:13:23,698 - lava_callback - DEBUG - Processing node 69bf6c51f1004e82292e420c: baseline-arm64-clabbe 2026-03-24 04:13:23,699 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf6c51f1004e82292e420c 2026-03-24 04:13:23,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 04:13:23,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf6c51f1004e82292e420c HTTP/1.1" 200 None 2026-03-24 04:13:23,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 04:13:23,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-24 04:13:23,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 04:13:23,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf6c51f1004e82292e420c HTTP/1.1" 200 None 2026-03-24 04:13:23,775 - lava_callback - ERROR - Error processing callback for node 69bf6c51f1004e82292e420c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40170 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46430 - "GET / HTTP/1.0" 200 OK 2026-03-24 04:15:08,945 - lava_callback - INFO - Received callback for node 69c1e3be0efaf21d931060a4 INFO: 172.21.0.1:46438 - "POST /node/69c1e3be0efaf21d931060a4 HTTP/1.0" 202 Accepted 2026-03-24 04:15:08,951 - lava_callback - INFO - Starting async job submit for node 69c1e3be0efaf21d931060a4 2026-03-24 04:15:08,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 04:15:08,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3be0efaf21d931060a4 HTTP/1.1" 200 None 2026-03-24 04:15:08,989 - lava_callback - DEBUG - Processing node 69c1e3be0efaf21d931060a4: baseline-arm64-broonie 2026-03-24 04:15:09,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 04:15:09,392 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 04:15:09,393 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c1e3be0efaf21d931060a4/log.txt.gz 2026-03-24 04:15:09,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 04:15:09,774 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 04:15:09,776 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c1e3be0efaf21d931060a4/lava-logs.yaml 2026-03-24 04:15:09,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 04:15:09,948 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 04:15:09,950 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c1e3be0efaf21d931060a4/lava_callback.json.gz 2026-03-24 04:15:09,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 04:15:09,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3be0efaf21d931060a4 HTTP/1.1" 200 None 2026-03-24 04:15:09,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 04:15:10,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-24 04:15:10,015 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 04:15:10,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e3be0efaf21d931060a4 HTTP/1.1" 200 None 2026-03-24 04:15:10,047 - lava_callback - INFO - Completed processing callback for node 69c1e3be0efaf21d931060a4 2026-03-24 04:15:11,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 04:15:11,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:37498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37508 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42402 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47524 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50710 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39230 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38136 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49876 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46608 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51692 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34210 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38910 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35834 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45214 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56112 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34798 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55860 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52000 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43914 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57748 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34576 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42460 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33590 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51024 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56022 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34416 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40644 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36648 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41384 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34584 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36158 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50454 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46422 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55994 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56620 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54346 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32778 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:32780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41344 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51564 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35414 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55974 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50592 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47804 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51794 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54704 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39770 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51972 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46386 - "GET / HTTP/1.0" 200 OK 2026-03-24 05:00:27,441 - lava_callback - INFO - Received callback for node 69bf775cf1004e82292e42ac INFO: 172.21.0.1:46394 - "POST /node/69bf775cf1004e82292e42ac HTTP/1.0" 202 Accepted 2026-03-24 05:00:27,447 - lava_callback - INFO - Starting async job submit for node 69bf775cf1004e82292e42ac 2026-03-24 05:00:27,448 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:00:27,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf775cf1004e82292e42ac HTTP/1.1" 200 None 2026-03-24 05:00:27,464 - lava_callback - DEBUG - Processing node 69bf775cf1004e82292e42ac: baseline-arm64-baylibre 2026-03-24 05:00:27,464 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf775cf1004e82292e42ac 2026-03-24 05:00:27,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:00:27,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf775cf1004e82292e42ac HTTP/1.1" 200 None 2026-03-24 05:00:27,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:00:27,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4e4a85a0f2cbb32a746 HTTP/1.1" 200 None 2026-03-24 05:00:27,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:00:27,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf775cf1004e82292e42ac HTTP/1.1" 200 None 2026-03-24 05:00:27,537 - lava_callback - ERROR - Error processing callback for node 69bf775cf1004e82292e42ac: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53098 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49150 - "GET / HTTP/1.0" 200 OK 2026-03-24 05:01:47,043 - lava_callback - INFO - Received callback for node 69bf77a8f1004e82292e42b5 INFO: 172.21.0.1:49162 - "POST /node/69bf77a8f1004e82292e42b5 HTTP/1.0" 202 Accepted 2026-03-24 05:01:47,049 - lava_callback - INFO - Starting async job submit for node 69bf77a8f1004e82292e42b5 2026-03-24 05:01:47,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:01:47,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf77a8f1004e82292e42b5 HTTP/1.1" 200 None 2026-03-24 05:01:47,065 - lava_callback - DEBUG - Processing node 69bf77a8f1004e82292e42b5: baseline-arm64-baylibre 2026-03-24 05:01:47,065 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf77a8f1004e82292e42b5 2026-03-24 05:01:47,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:01:47,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf77a8f1004e82292e42b5 HTTP/1.1" 200 None 2026-03-24 05:01:47,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:01:47,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4e4a85a0f2cbb32a746 HTTP/1.1" 200 None 2026-03-24 05:01:47,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:01:47,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf77a8f1004e82292e42b5 HTTP/1.1" 200 None 2026-03-24 05:01:47,127 - lava_callback - ERROR - Error processing callback for node 69bf77a8f1004e82292e42b5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51642 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33616 - "GET / HTTP/1.0" 200 OK 2026-03-24 05:02:47,037 - lava_callback - INFO - Received callback for node 69bf77f3f1004e82292e42c3 INFO: 172.21.0.1:33630 - "POST /node/69bf77f3f1004e82292e42c3 HTTP/1.0" 202 Accepted 2026-03-24 05:02:47,042 - lava_callback - INFO - Starting async job submit for node 69bf77f3f1004e82292e42c3 2026-03-24 05:02:47,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:02:47,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf77f3f1004e82292e42c3 HTTP/1.1" 200 None 2026-03-24 05:02:47,059 - lava_callback - DEBUG - Processing node 69bf77f3f1004e82292e42c3: baseline-arm64-baylibre 2026-03-24 05:02:47,059 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf77f3f1004e82292e42c3 2026-03-24 05:02:47,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:02:47,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf77f3f1004e82292e42c3 HTTP/1.1" 200 None 2026-03-24 05:02:47,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:02:47,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4e4a85a0f2cbb32a746 HTTP/1.1" 200 None 2026-03-24 05:02:47,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:02:47,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf77f3f1004e82292e42c3 HTTP/1.1" 200 None 2026-03-24 05:02:47,146 - lava_callback - ERROR - Error processing callback for node 69bf77f3f1004e82292e42c3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45096 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57270 - "GET / HTTP/1.0" 200 OK 2026-03-24 05:04:07,167 - lava_callback - INFO - Received callback for node 69bf783ef1004e82292e42cd INFO: 172.21.0.1:57272 - "POST /node/69bf783ef1004e82292e42cd HTTP/1.0" 202 Accepted 2026-03-24 05:04:07,173 - lava_callback - INFO - Starting async job submit for node 69bf783ef1004e82292e42cd 2026-03-24 05:04:07,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:04:07,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf783ef1004e82292e42cd HTTP/1.1" 200 None 2026-03-24 05:04:07,189 - lava_callback - DEBUG - Processing node 69bf783ef1004e82292e42cd: baseline-arm64-baylibre 2026-03-24 05:04:07,189 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf783ef1004e82292e42cd 2026-03-24 05:04:07,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:04:07,203 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf783ef1004e82292e42cd HTTP/1.1" 200 None 2026-03-24 05:04:07,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:04:07,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4e4a85a0f2cbb32a746 HTTP/1.1" 200 None 2026-03-24 05:04:07,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:04:07,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf783ef1004e82292e42cd HTTP/1.1" 200 None 2026-03-24 05:04:07,244 - lava_callback - ERROR - Error processing callback for node 69bf783ef1004e82292e42cd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60910 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37642 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53110 - "GET / HTTP/1.0" 200 OK 2026-03-24 05:05:27,254 - lava_callback - INFO - Received callback for node 69bf788af1004e82292e42db INFO: 172.21.0.1:53120 - "POST /node/69bf788af1004e82292e42db HTTP/1.0" 202 Accepted 2026-03-24 05:05:27,260 - lava_callback - INFO - Starting async job submit for node 69bf788af1004e82292e42db 2026-03-24 05:05:27,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:05:27,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf788af1004e82292e42db HTTP/1.1" 200 None 2026-03-24 05:05:27,276 - lava_callback - DEBUG - Processing node 69bf788af1004e82292e42db: baseline-arm64-baylibre 2026-03-24 05:05:27,277 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf788af1004e82292e42db 2026-03-24 05:05:27,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:05:27,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf788af1004e82292e42db HTTP/1.1" 200 None 2026-03-24 05:05:27,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:05:27,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4e4a85a0f2cbb32a746 HTTP/1.1" 200 None 2026-03-24 05:05:27,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:05:27,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf788af1004e82292e42db HTTP/1.1" 200 None 2026-03-24 05:05:27,332 - lava_callback - ERROR - Error processing callback for node 69bf788af1004e82292e42db: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44216 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39620 - "GET / HTTP/1.0" 200 OK 2026-03-24 05:06:47,362 - lava_callback - INFO - Received callback for node 69bf78d5f1004e82292e42e7 INFO: 172.21.0.1:39634 - "POST /node/69bf78d5f1004e82292e42e7 HTTP/1.0" 202 Accepted 2026-03-24 05:06:47,368 - lava_callback - INFO - Starting async job submit for node 69bf78d5f1004e82292e42e7 2026-03-24 05:06:47,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:06:47,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf78d5f1004e82292e42e7 HTTP/1.1" 200 None 2026-03-24 05:06:47,384 - lava_callback - DEBUG - Processing node 69bf78d5f1004e82292e42e7: baseline-arm64-baylibre 2026-03-24 05:06:47,385 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf78d5f1004e82292e42e7 2026-03-24 05:06:47,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:06:47,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf78d5f1004e82292e42e7 HTTP/1.1" 200 None 2026-03-24 05:06:47,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:06:47,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4e4a85a0f2cbb32a746 HTTP/1.1" 200 None 2026-03-24 05:06:47,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:06:47,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf78d5f1004e82292e42e7 HTTP/1.1" 200 None 2026-03-24 05:06:47,445 - lava_callback - ERROR - Error processing callback for node 69bf78d5f1004e82292e42e7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45926 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32908 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:32914 - "GET / HTTP/1.0" 200 OK 2026-03-24 05:08:26,041 - lava_callback - INFO - Received callback for node 69bf7943f1004e82292e42fa INFO: 172.21.0.1:44442 - "POST /node/69bf7943f1004e82292e42fa HTTP/1.0" 202 Accepted 2026-03-24 05:08:26,046 - lava_callback - INFO - Starting async job submit for node 69bf7943f1004e82292e42fa 2026-03-24 05:08:26,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:08:26,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf7943f1004e82292e42fa HTTP/1.1" 200 None 2026-03-24 05:08:26,067 - lava_callback - DEBUG - Processing node 69bf7943f1004e82292e42fa: baseline-arm64-clabbe 2026-03-24 05:08:26,068 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf7943f1004e82292e42fa 2026-03-24 05:08:26,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:08:26,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf7943f1004e82292e42fa HTTP/1.1" 200 None 2026-03-24 05:08:26,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:08:26,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4e4a85a0f2cbb32a746 HTTP/1.1" 200 None 2026-03-24 05:08:26,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:08:26,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf7943f1004e82292e42fa HTTP/1.1" 200 None 2026-03-24 05:08:26,125 - lava_callback - ERROR - Error processing callback for node 69bf7943f1004e82292e42fa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59992 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32908 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:32910 - "GET / HTTP/1.0" 200 OK 2026-03-24 05:10:26,116 - lava_callback - INFO - Received callback for node 69bf79bdf1004e82292e430a INFO: 172.21.0.1:47814 - "POST /node/69bf79bdf1004e82292e430a HTTP/1.0" 202 Accepted 2026-03-24 05:10:26,122 - lava_callback - INFO - Starting async job submit for node 69bf79bdf1004e82292e430a 2026-03-24 05:10:26,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:10:26,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf79bdf1004e82292e430a HTTP/1.1" 200 None 2026-03-24 05:10:26,140 - lava_callback - DEBUG - Processing node 69bf79bdf1004e82292e430a: baseline-arm64-clabbe 2026-03-24 05:10:26,140 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf79bdf1004e82292e430a 2026-03-24 05:10:26,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:10:26,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf79bdf1004e82292e430a HTTP/1.1" 200 None 2026-03-24 05:10:26,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:10:26,183 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4e4a85a0f2cbb32a746 HTTP/1.1" 200 None 2026-03-24 05:10:26,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:10:26,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf79bdf1004e82292e430a HTTP/1.1" 200 None 2026-03-24 05:10:26,211 - lava_callback - ERROR - Error processing callback for node 69bf79bdf1004e82292e430a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50000 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46696 - "GET / HTTP/1.0" 200 OK 2026-03-24 05:11:46,136 - lava_callback - INFO - Received callback for node 69bf7a08f1004e82292e431d INFO: 172.21.0.1:36954 - "POST /node/69bf7a08f1004e82292e431d HTTP/1.0" 202 Accepted 2026-03-24 05:11:46,141 - lava_callback - INFO - Starting async job submit for node 69bf7a08f1004e82292e431d 2026-03-24 05:11:46,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:11:46,157 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf7a08f1004e82292e431d HTTP/1.1" 200 None 2026-03-24 05:11:46,158 - lava_callback - DEBUG - Processing node 69bf7a08f1004e82292e431d: baseline-arm64-clabbe 2026-03-24 05:11:46,158 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf7a08f1004e82292e431d 2026-03-24 05:11:46,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:11:46,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf7a08f1004e82292e431d HTTP/1.1" 200 None 2026-03-24 05:11:46,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:11:46,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4e4a85a0f2cbb32a746 HTTP/1.1" 200 None 2026-03-24 05:11:46,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:11:46,236 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf7a08f1004e82292e431d HTTP/1.1" 200 None 2026-03-24 05:11:46,237 - lava_callback - ERROR - Error processing callback for node 69bf7a08f1004e82292e431d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49892 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40338 - "GET / HTTP/1.0" 200 OK 2026-03-24 05:13:06,288 - lava_callback - INFO - Received callback for node 69bf7a52f1004e82292e4328 INFO: 172.21.0.1:46488 - "POST /node/69bf7a52f1004e82292e4328 HTTP/1.0" 202 Accepted 2026-03-24 05:13:06,294 - lava_callback - INFO - Starting async job submit for node 69bf7a52f1004e82292e4328 2026-03-24 05:13:06,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:13:06,310 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf7a52f1004e82292e4328 HTTP/1.1" 200 None 2026-03-24 05:13:06,311 - lava_callback - DEBUG - Processing node 69bf7a52f1004e82292e4328: baseline-arm64-clabbe 2026-03-24 05:13:06,311 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf7a52f1004e82292e4328 2026-03-24 05:13:06,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:13:06,325 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf7a52f1004e82292e4328 HTTP/1.1" 200 None 2026-03-24 05:13:06,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:13:06,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4e4a85a0f2cbb32a746 HTTP/1.1" 200 None 2026-03-24 05:13:06,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:13:06,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf7a52f1004e82292e4328 HTTP/1.1" 200 None 2026-03-24 05:13:06,382 - lava_callback - ERROR - Error processing callback for node 69bf7a52f1004e82292e4328: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42968 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54640 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54656 - "GET / HTTP/1.0" 200 OK 2026-03-24 05:14:26,296 - lava_callback - INFO - Received callback for node 69bf7a9df1004e82292e433e INFO: 172.21.0.1:60876 - "POST /node/69bf7a9df1004e82292e433e HTTP/1.0" 202 Accepted 2026-03-24 05:14:26,301 - lava_callback - INFO - Starting async job submit for node 69bf7a9df1004e82292e433e 2026-03-24 05:14:26,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:14:26,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf7a9df1004e82292e433e HTTP/1.1" 200 None 2026-03-24 05:14:26,318 - lava_callback - DEBUG - Processing node 69bf7a9df1004e82292e433e: baseline-arm64-clabbe 2026-03-24 05:14:26,319 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf7a9df1004e82292e433e 2026-03-24 05:14:26,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:14:26,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf7a9df1004e82292e433e HTTP/1.1" 200 None 2026-03-24 05:14:26,334 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:14:26,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4e4a85a0f2cbb32a746 HTTP/1.1" 200 None 2026-03-24 05:14:26,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:14:26,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf7a9df1004e82292e433e HTTP/1.1" 200 None 2026-03-24 05:14:26,374 - lava_callback - ERROR - Error processing callback for node 69bf7a9df1004e82292e433e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46132 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46138 - "GET / HTTP/1.0" 200 OK 2026-03-24 05:15:26,289 - lava_callback - INFO - Received callback for node 69bf7ae9f1004e82292e4348 INFO: 172.21.0.1:55180 - "POST /node/69bf7ae9f1004e82292e4348 HTTP/1.0" 202 Accepted 2026-03-24 05:15:26,295 - lava_callback - INFO - Starting async job submit for node 69bf7ae9f1004e82292e4348 2026-03-24 05:15:26,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:15:26,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf7ae9f1004e82292e4348 HTTP/1.1" 200 None 2026-03-24 05:15:26,320 - lava_callback - DEBUG - Processing node 69bf7ae9f1004e82292e4348: baseline-arm64-clabbe 2026-03-24 05:15:26,320 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf7ae9f1004e82292e4348 2026-03-24 05:15:26,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:15:26,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf7ae9f1004e82292e4348 HTTP/1.1" 200 None 2026-03-24 05:15:26,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:15:26,351 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4e4a85a0f2cbb32a746 HTTP/1.1" 200 None 2026-03-24 05:15:26,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:15:26,374 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf7ae9f1004e82292e4348 HTTP/1.1" 200 None 2026-03-24 05:15:26,375 - lava_callback - ERROR - Error processing callback for node 69bf7ae9f1004e82292e4348: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50246 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45658 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33530 - "GET / HTTP/1.0" 200 OK 2026-03-24 05:17:47,974 - lava_callback - INFO - Received callback for node 69bf7b66f1004e82292e436e INFO: 172.21.0.1:33538 - "POST /node/69bf7b66f1004e82292e436e HTTP/1.0" 202 Accepted 2026-03-24 05:17:47,979 - lava_callback - INFO - Starting async job submit for node 69bf7b66f1004e82292e436e 2026-03-24 05:17:47,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:17:47,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf7b66f1004e82292e436e HTTP/1.1" 200 None 2026-03-24 05:17:47,995 - lava_callback - DEBUG - Processing node 69bf7b66f1004e82292e436e: baseline-arm-baylibre 2026-03-24 05:17:47,995 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf7b66f1004e82292e436e 2026-03-24 05:17:47,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:17:48,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf7b66f1004e82292e436e HTTP/1.1" 200 None 2026-03-24 05:17:48,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:17:48,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4d5a85a0f2cbb32a731 HTTP/1.1" 200 None 2026-03-24 05:17:48,029 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:17:48,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf7b66f1004e82292e436e HTTP/1.1" 200 None 2026-03-24 05:17:48,053 - lava_callback - ERROR - Error processing callback for node 69bf7b66f1004e82292e436e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33402 - "GET / HTTP/1.0" 200 OK 2026-03-24 05:18:07,883 - lava_callback - INFO - Received callback for node 69bf7b8af1004e82292e4379 INFO: 172.21.0.1:33408 - "POST /node/69bf7b8af1004e82292e4379 HTTP/1.0" 202 Accepted 2026-03-24 05:18:07,889 - lava_callback - INFO - Starting async job submit for node 69bf7b8af1004e82292e4379 2026-03-24 05:18:07,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:18:07,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf7b8af1004e82292e4379 HTTP/1.1" 200 None 2026-03-24 05:18:07,913 - lava_callback - DEBUG - Processing node 69bf7b8af1004e82292e4379: baseline-arm-baylibre 2026-03-24 05:18:07,913 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf7b8af1004e82292e4379 2026-03-24 05:18:07,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:18:07,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf7b8af1004e82292e4379 HTTP/1.1" 200 None 2026-03-24 05:18:07,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:18:07,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4d5a85a0f2cbb32a731 HTTP/1.1" 200 None 2026-03-24 05:18:07,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:18:07,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf7b8af1004e82292e4379 HTTP/1.1" 200 None 2026-03-24 05:18:07,969 - lava_callback - ERROR - Error processing callback for node 69bf7b8af1004e82292e4379: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47098 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39384 - "GET / HTTP/1.0" 200 OK 2026-03-24 05:18:47,938 - lava_callback - INFO - Received callback for node 69bf7bb0f1004e82292e4384 INFO: 172.21.0.1:39400 - "POST /node/69bf7bb0f1004e82292e4384 HTTP/1.0" 202 Accepted 2026-03-24 05:18:47,943 - lava_callback - INFO - Starting async job submit for node 69bf7bb0f1004e82292e4384 2026-03-24 05:18:47,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:18:47,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf7bb0f1004e82292e4384 HTTP/1.1" 200 None 2026-03-24 05:18:47,959 - lava_callback - DEBUG - Processing node 69bf7bb0f1004e82292e4384: baseline-arm-baylibre 2026-03-24 05:18:47,959 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf7bb0f1004e82292e4384 2026-03-24 05:18:47,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:18:47,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf7bb0f1004e82292e4384 HTTP/1.1" 200 None 2026-03-24 05:18:47,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:18:47,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4d5a85a0f2cbb32a731 HTTP/1.1" 200 None 2026-03-24 05:18:47,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:18:48,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf7bb0f1004e82292e4384 HTTP/1.1" 200 None 2026-03-24 05:18:48,027 - lava_callback - ERROR - Error processing callback for node 69bf7bb0f1004e82292e4384: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43764 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47976 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60558 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33552 - "GET / HTTP/1.0" 200 OK 2026-03-24 05:21:46,598 - lava_callback - INFO - Received callback for node 69bf7c67f1004e82292e43ab INFO: 172.21.0.1:33560 - "POST /node/69bf7c67f1004e82292e43ab HTTP/1.0" 202 Accepted 2026-03-24 05:21:46,603 - lava_callback - INFO - Starting async job submit for node 69bf7c67f1004e82292e43ab 2026-03-24 05:21:46,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:21:46,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf7c67f1004e82292e43ab HTTP/1.1" 200 None 2026-03-24 05:21:46,620 - lava_callback - DEBUG - Processing node 69bf7c67f1004e82292e43ab: baseline-arm-clabbe 2026-03-24 05:21:46,620 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf7c67f1004e82292e43ab 2026-03-24 05:21:46,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:21:46,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf7c67f1004e82292e43ab HTTP/1.1" 200 None 2026-03-24 05:21:46,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:21:46,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4d5a85a0f2cbb32a731 HTTP/1.1" 200 None 2026-03-24 05:21:46,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:21:46,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf7c67f1004e82292e43ab HTTP/1.1" 200 None 2026-03-24 05:21:46,678 - lava_callback - ERROR - Error processing callback for node 69bf7c67f1004e82292e43ab: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45790 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40210 - "GET / HTTP/1.0" 200 OK 2026-03-24 05:22:26,604 - lava_callback - INFO - Received callback for node 69bf7c8cf1004e82292e43bb INFO: 172.21.0.1:40224 - "POST /node/69bf7c8cf1004e82292e43bb HTTP/1.0" 202 Accepted 2026-03-24 05:22:26,610 - lava_callback - INFO - Starting async job submit for node 69bf7c8cf1004e82292e43bb 2026-03-24 05:22:26,611 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:22:26,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf7c8cf1004e82292e43bb HTTP/1.1" 200 None 2026-03-24 05:22:26,626 - lava_callback - DEBUG - Processing node 69bf7c8cf1004e82292e43bb: baseline-arm-clabbe 2026-03-24 05:22:26,626 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf7c8cf1004e82292e43bb 2026-03-24 05:22:26,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:22:26,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf7c8cf1004e82292e43bb HTTP/1.1" 200 None 2026-03-24 05:22:26,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:22:26,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4d5a85a0f2cbb32a731 HTTP/1.1" 200 None 2026-03-24 05:22:26,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:22:26,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf7c8cf1004e82292e43bb HTTP/1.1" 200 None 2026-03-24 05:22:26,677 - lava_callback - ERROR - Error processing callback for node 69bf7c8cf1004e82292e43bb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59172 - "GET / HTTP/1.0" 200 OK 2026-03-24 05:23:06,653 - lava_callback - INFO - Received callback for node 69bf7cb1f1004e82292e43c4 INFO: 172.21.0.1:59174 - "POST /node/69bf7cb1f1004e82292e43c4 HTTP/1.0" 202 Accepted 2026-03-24 05:23:06,659 - lava_callback - INFO - Starting async job submit for node 69bf7cb1f1004e82292e43c4 2026-03-24 05:23:06,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:23:06,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf7cb1f1004e82292e43c4 HTTP/1.1" 200 None 2026-03-24 05:23:06,681 - lava_callback - DEBUG - Processing node 69bf7cb1f1004e82292e43c4: baseline-arm-clabbe 2026-03-24 05:23:06,681 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf7cb1f1004e82292e43c4 2026-03-24 05:23:06,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:23:06,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf7cb1f1004e82292e43c4 HTTP/1.1" 200 None 2026-03-24 05:23:06,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:23:06,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4d5a85a0f2cbb32a731 HTTP/1.1" 200 None 2026-03-24 05:23:06,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:23:06,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf7cb1f1004e82292e43c4 HTTP/1.1" 200 None 2026-03-24 05:23:06,741 - lava_callback - ERROR - Error processing callback for node 69bf7cb1f1004e82292e43c4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44084 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51942 - "GET / HTTP/1.0" 200 OK 2026-03-24 05:23:46,648 - lava_callback - INFO - Received callback for node 69bf7cd6f1004e82292e43d1 INFO: 172.21.0.1:51950 - "POST /node/69bf7cd6f1004e82292e43d1 HTTP/1.0" 202 Accepted 2026-03-24 05:23:46,653 - lava_callback - INFO - Starting async job submit for node 69bf7cd6f1004e82292e43d1 2026-03-24 05:23:46,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:23:46,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf7cd6f1004e82292e43d1 HTTP/1.1" 200 None 2026-03-24 05:23:46,682 - lava_callback - DEBUG - Processing node 69bf7cd6f1004e82292e43d1: baseline-arm-clabbe 2026-03-24 05:23:46,682 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf7cd6f1004e82292e43d1 2026-03-24 05:23:46,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:23:46,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf7cd6f1004e82292e43d1 HTTP/1.1" 200 None 2026-03-24 05:23:46,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:23:46,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4d5a85a0f2cbb32a731 HTTP/1.1" 200 None 2026-03-24 05:23:46,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:23:46,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf7cd6f1004e82292e43d1 HTTP/1.1" 200 None 2026-03-24 05:23:46,740 - lava_callback - ERROR - Error processing callback for node 69bf7cd6f1004e82292e43d1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57602 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39066 - "GET / HTTP/1.0" 200 OK 2026-03-24 05:24:26,717 - lava_callback - INFO - Received callback for node 69bf7cfbf1004e82292e43e1 INFO: 172.21.0.1:39076 - "POST /node/69bf7cfbf1004e82292e43e1 HTTP/1.0" 202 Accepted 2026-03-24 05:24:26,722 - lava_callback - INFO - Starting async job submit for node 69bf7cfbf1004e82292e43e1 2026-03-24 05:24:26,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:24:26,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf7cfbf1004e82292e43e1 HTTP/1.1" 200 None 2026-03-24 05:24:26,743 - lava_callback - DEBUG - Processing node 69bf7cfbf1004e82292e43e1: baseline-arm-clabbe 2026-03-24 05:24:26,743 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf7cfbf1004e82292e43e1 2026-03-24 05:24:26,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:24:26,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf7cfbf1004e82292e43e1 HTTP/1.1" 200 None 2026-03-24 05:24:26,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:24:26,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4d5a85a0f2cbb32a731 HTTP/1.1" 200 None 2026-03-24 05:24:26,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:24:26,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf7cfbf1004e82292e43e1 HTTP/1.1" 200 None 2026-03-24 05:24:26,794 - lava_callback - ERROR - Error processing callback for node 69bf7cfbf1004e82292e43e1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40818 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45262 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35868 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47720 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43124 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58644 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54762 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50214 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49752 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42530 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40010 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34256 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45426 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54232 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52154 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39930 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56352 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60234 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55196 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42152 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59558 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42882 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33588 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50318 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52662 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43870 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60014 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48688 - "GET / HTTP/1.0" 200 OK 2026-03-24 05:52:09,848 - lava_callback - INFO - Received callback for node 69bf8386f1004e82292e4427 INFO: 172.21.0.1:56684 - "POST /node/69bf8386f1004e82292e4427 HTTP/1.0" 202 Accepted 2026-03-24 05:52:09,854 - lava_callback - INFO - Starting async job submit for node 69bf8386f1004e82292e4427 2026-03-24 05:52:09,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:52:09,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf8386f1004e82292e4427 HTTP/1.1" 200 None 2026-03-24 05:52:09,869 - lava_callback - DEBUG - Processing node 69bf8386f1004e82292e4427: baseline-arm-baylibre 2026-03-24 05:52:09,869 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf8386f1004e82292e4427 2026-03-24 05:52:09,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:52:09,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf8386f1004e82292e4427 HTTP/1.1" 200 None 2026-03-24 05:52:09,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:52:09,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21a117e5cb936602bc16 HTTP/1.1" 200 None 2026-03-24 05:52:09,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:52:09,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf8386f1004e82292e4427 HTTP/1.1" 200 None 2026-03-24 05:52:09,946 - lava_callback - ERROR - Error processing callback for node 69bf8386f1004e82292e4427: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56704 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36764 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36716 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52790 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54284 - "GET / HTTP/1.0" 200 OK 2026-03-24 05:55:27,993 - lava_callback - INFO - Received callback for node 69bf843ff1004e82292e4433 INFO: 172.21.0.1:54286 - "POST /node/69bf843ff1004e82292e4433 HTTP/1.0" 202 Accepted 2026-03-24 05:55:27,999 - lava_callback - INFO - Starting async job submit for node 69bf843ff1004e82292e4433 2026-03-24 05:55:27,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:55:28,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf843ff1004e82292e4433 HTTP/1.1" 200 None 2026-03-24 05:55:28,015 - lava_callback - DEBUG - Processing node 69bf843ff1004e82292e4433: baseline-arm-clabbe 2026-03-24 05:55:28,015 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf843ff1004e82292e4433 2026-03-24 05:55:28,016 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:55:28,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf843ff1004e82292e4433 HTTP/1.1" 200 None 2026-03-24 05:55:28,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:55:28,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21a117e5cb936602bc16 HTTP/1.1" 200 None 2026-03-24 05:55:28,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:55:28,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf843ff1004e82292e4433 HTTP/1.1" 200 None 2026-03-24 05:55:28,076 - lava_callback - ERROR - Error processing callback for node 69bf843ff1004e82292e4433: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40508 - "GET / HTTP/1.0" 200 OK 2026-03-24 05:56:07,922 - lava_callback - INFO - Received callback for node 69bf8468f1004e82292e443e INFO: 172.21.0.1:40524 - "POST /node/69bf8468f1004e82292e443e HTTP/1.0" 202 Accepted 2026-03-24 05:56:07,927 - lava_callback - INFO - Starting async job submit for node 69bf8468f1004e82292e443e 2026-03-24 05:56:07,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:56:07,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf8468f1004e82292e443e HTTP/1.1" 200 None 2026-03-24 05:56:07,945 - lava_callback - DEBUG - Processing node 69bf8468f1004e82292e443e: baseline-arm-clabbe 2026-03-24 05:56:07,945 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf8468f1004e82292e443e 2026-03-24 05:56:07,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:56:07,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf8468f1004e82292e443e HTTP/1.1" 200 None 2026-03-24 05:56:07,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:56:08,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21a117e5cb936602bc16 HTTP/1.1" 200 None 2026-03-24 05:56:08,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:56:08,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf8468f1004e82292e443e HTTP/1.1" 200 None 2026-03-24 05:56:08,036 - lava_callback - ERROR - Error processing callback for node 69bf8468f1004e82292e443e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57242 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45208 - "GET / HTTP/1.0" 200 OK 2026-03-24 05:56:48,080 - lava_callback - INFO - Received callback for node 69bf848df1004e82292e444c INFO: 172.21.0.1:45216 - "POST /node/69bf848df1004e82292e444c HTTP/1.0" 202 Accepted 2026-03-24 05:56:48,085 - lava_callback - INFO - Starting async job submit for node 69bf848df1004e82292e444c 2026-03-24 05:56:48,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:56:48,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf848df1004e82292e444c HTTP/1.1" 200 None 2026-03-24 05:56:48,103 - lava_callback - DEBUG - Processing node 69bf848df1004e82292e444c: baseline-arm-clabbe 2026-03-24 05:56:48,103 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf848df1004e82292e444c 2026-03-24 05:56:48,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:56:48,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf848df1004e82292e444c HTTP/1.1" 200 None 2026-03-24 05:56:48,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:56:48,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21a117e5cb936602bc16 HTTP/1.1" 200 None 2026-03-24 05:56:48,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:56:48,158 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf848df1004e82292e444c HTTP/1.1" 200 None 2026-03-24 05:56:48,159 - lava_callback - ERROR - Error processing callback for node 69bf848df1004e82292e444c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55582 - "GET / HTTP/1.0" 200 OK 2026-03-24 05:57:07,994 - lava_callback - INFO - Received callback for node 69bf84b1f1004e82292e4459 INFO: 172.21.0.1:55588 - "POST /node/69bf84b1f1004e82292e4459 HTTP/1.0" 202 Accepted 2026-03-24 05:57:07,999 - lava_callback - INFO - Starting async job submit for node 69bf84b1f1004e82292e4459 2026-03-24 05:57:08,000 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:57:08,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf84b1f1004e82292e4459 HTTP/1.1" 200 None 2026-03-24 05:57:08,035 - lava_callback - DEBUG - Processing node 69bf84b1f1004e82292e4459: baseline-arm-clabbe 2026-03-24 05:57:08,035 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf84b1f1004e82292e4459 2026-03-24 05:57:08,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:57:08,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf84b1f1004e82292e4459 HTTP/1.1" 200 None 2026-03-24 05:57:08,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:57:08,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21a117e5cb936602bc16 HTTP/1.1" 200 None 2026-03-24 05:57:08,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:57:08,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf84b1f1004e82292e4459 HTTP/1.1" 200 None 2026-03-24 05:57:08,121 - lava_callback - ERROR - Error processing callback for node 69bf84b1f1004e82292e4459: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51656 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34240 - "GET / HTTP/1.0" 200 OK 2026-03-24 05:57:48,091 - lava_callback - INFO - Received callback for node 69bf84d5f1004e82292e4468 INFO: 172.21.0.1:34248 - "POST /node/69bf84d5f1004e82292e4468 HTTP/1.0" 202 Accepted 2026-03-24 05:57:48,097 - lava_callback - INFO - Starting async job submit for node 69bf84d5f1004e82292e4468 2026-03-24 05:57:48,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:57:48,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf84d5f1004e82292e4468 HTTP/1.1" 200 None 2026-03-24 05:57:48,113 - lava_callback - DEBUG - Processing node 69bf84d5f1004e82292e4468: baseline-arm-clabbe 2026-03-24 05:57:48,113 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf84d5f1004e82292e4468 2026-03-24 05:57:48,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:57:48,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf84d5f1004e82292e4468 HTTP/1.1" 200 None 2026-03-24 05:57:48,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:57:48,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21a117e5cb936602bc16 HTTP/1.1" 200 None 2026-03-24 05:57:48,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:57:48,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf84d5f1004e82292e4468 HTTP/1.1" 200 None 2026-03-24 05:57:48,176 - lava_callback - ERROR - Error processing callback for node 69bf84d5f1004e82292e4468: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35714 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52452 - "GET / HTTP/1.0" 200 OK 2026-03-24 05:58:28,081 - lava_callback - INFO - Received callback for node 69bf84f9f1004e82292e4476 INFO: 172.21.0.1:52464 - "POST /node/69bf84f9f1004e82292e4476 HTTP/1.0" 202 Accepted 2026-03-24 05:58:28,086 - lava_callback - INFO - Starting async job submit for node 69bf84f9f1004e82292e4476 2026-03-24 05:58:28,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:58:28,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf84f9f1004e82292e4476 HTTP/1.1" 200 None 2026-03-24 05:58:28,103 - lava_callback - DEBUG - Processing node 69bf84f9f1004e82292e4476: baseline-arm-clabbe 2026-03-24 05:58:28,103 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf84f9f1004e82292e4476 2026-03-24 05:58:28,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:58:28,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf84f9f1004e82292e4476 HTTP/1.1" 200 None 2026-03-24 05:58:28,119 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:58:28,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21a117e5cb936602bc16 HTTP/1.1" 200 None 2026-03-24 05:58:28,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:58:28,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf84f9f1004e82292e4476 HTTP/1.1" 200 None 2026-03-24 05:58:28,165 - lava_callback - ERROR - Error processing callback for node 69bf84f9f1004e82292e4476: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47372 - "GET / HTTP/1.0" 200 OK 2026-03-24 05:59:08,167 - lava_callback - INFO - Received callback for node 69bf851df1004e82292e4484 INFO: 172.21.0.1:47382 - "POST /node/69bf851df1004e82292e4484 HTTP/1.0" 202 Accepted 2026-03-24 05:59:08,173 - lava_callback - INFO - Starting async job submit for node 69bf851df1004e82292e4484 2026-03-24 05:59:08,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:59:08,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf851df1004e82292e4484 HTTP/1.1" 200 None 2026-03-24 05:59:08,199 - lava_callback - DEBUG - Processing node 69bf851df1004e82292e4484: baseline-arm-clabbe 2026-03-24 05:59:08,199 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf851df1004e82292e4484 2026-03-24 05:59:08,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:59:08,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf851df1004e82292e4484 HTTP/1.1" 200 None 2026-03-24 05:59:08,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:59:08,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21a117e5cb936602bc16 HTTP/1.1" 200 None 2026-03-24 05:59:08,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:59:08,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf851df1004e82292e4484 HTTP/1.1" 200 None 2026-03-24 05:59:08,250 - lava_callback - ERROR - Error processing callback for node 69bf851df1004e82292e4484: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52014 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34778 - "GET / HTTP/1.0" 200 OK 2026-03-24 05:59:48,135 - lava_callback - INFO - Received callback for node 69bf8541f1004e82292e4490 INFO: 172.21.0.1:34784 - "POST /node/69bf8541f1004e82292e4490 HTTP/1.0" 202 Accepted 2026-03-24 05:59:48,141 - lava_callback - INFO - Starting async job submit for node 69bf8541f1004e82292e4490 2026-03-24 05:59:48,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:59:48,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf8541f1004e82292e4490 HTTP/1.1" 200 None 2026-03-24 05:59:48,163 - lava_callback - DEBUG - Processing node 69bf8541f1004e82292e4490: baseline-arm-clabbe 2026-03-24 05:59:48,163 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf8541f1004e82292e4490 2026-03-24 05:59:48,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:59:48,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf8541f1004e82292e4490 HTTP/1.1" 200 None 2026-03-24 05:59:48,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:59:48,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21a117e5cb936602bc16 HTTP/1.1" 200 None 2026-03-24 05:59:48,196 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 05:59:48,220 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf8541f1004e82292e4490 HTTP/1.1" 200 None 2026-03-24 05:59:48,222 - lava_callback - ERROR - Error processing callback for node 69bf8541f1004e82292e4490: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57898 - "GET / HTTP/1.0" 200 OK 2026-03-24 06:00:08,334 - lava_callback - INFO - Received callback for node 69bf8564f1004e82292e449f INFO: 172.21.0.1:57914 - "POST /node/69bf8564f1004e82292e449f HTTP/1.0" 202 Accepted 2026-03-24 06:00:08,340 - lava_callback - INFO - Starting async job submit for node 69bf8564f1004e82292e449f 2026-03-24 06:00:08,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:00:08,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf8564f1004e82292e449f HTTP/1.1" 200 None 2026-03-24 06:00:08,355 - lava_callback - DEBUG - Processing node 69bf8564f1004e82292e449f: baseline-arm-clabbe 2026-03-24 06:00:08,355 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf8564f1004e82292e449f 2026-03-24 06:00:08,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:00:08,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf8564f1004e82292e449f HTTP/1.1" 200 None 2026-03-24 06:00:08,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:00:08,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21a117e5cb936602bc16 HTTP/1.1" 200 None 2026-03-24 06:00:08,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:00:08,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf8564f1004e82292e449f HTTP/1.1" 200 None 2026-03-24 06:00:08,410 - lava_callback - ERROR - Error processing callback for node 69bf8564f1004e82292e449f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54950 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50216 - "GET / HTTP/1.0" 200 OK 2026-03-24 06:00:48,203 - lava_callback - INFO - Received callback for node 69bf8589f1004e82292e44ae INFO: 172.21.0.1:50232 - "POST /node/69bf8589f1004e82292e44ae HTTP/1.0" 202 Accepted 2026-03-24 06:00:48,208 - lava_callback - INFO - Starting async job submit for node 69bf8589f1004e82292e44ae 2026-03-24 06:00:48,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:00:48,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf8589f1004e82292e44ae HTTP/1.1" 200 None 2026-03-24 06:00:48,243 - lava_callback - DEBUG - Processing node 69bf8589f1004e82292e44ae: baseline-arm-clabbe 2026-03-24 06:00:48,243 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf8589f1004e82292e44ae 2026-03-24 06:00:48,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:00:48,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf8589f1004e82292e44ae HTTP/1.1" 200 None 2026-03-24 06:00:48,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:00:48,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21a117e5cb936602bc16 HTTP/1.1" 200 None 2026-03-24 06:00:48,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:00:48,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf8589f1004e82292e44ae HTTP/1.1" 200 None 2026-03-24 06:00:48,296 - lava_callback - ERROR - Error processing callback for node 69bf8589f1004e82292e44ae: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52766 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49876 - "GET / HTTP/1.0" 200 OK 2026-03-24 06:01:28,287 - lava_callback - INFO - Received callback for node 69bf85adf1004e82292e44b9 INFO: 172.21.0.1:49886 - "POST /node/69bf85adf1004e82292e44b9 HTTP/1.0" 202 Accepted 2026-03-24 06:01:28,293 - lava_callback - INFO - Starting async job submit for node 69bf85adf1004e82292e44b9 2026-03-24 06:01:28,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:01:28,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf85adf1004e82292e44b9 HTTP/1.1" 200 None 2026-03-24 06:01:28,310 - lava_callback - DEBUG - Processing node 69bf85adf1004e82292e44b9: baseline-arm-clabbe 2026-03-24 06:01:28,310 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf85adf1004e82292e44b9 2026-03-24 06:01:28,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:01:28,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf85adf1004e82292e44b9 HTTP/1.1" 200 None 2026-03-24 06:01:28,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:01:28,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21a117e5cb936602bc16 HTTP/1.1" 200 None 2026-03-24 06:01:28,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:01:28,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf85adf1004e82292e44b9 HTTP/1.1" 200 None 2026-03-24 06:01:28,366 - lava_callback - ERROR - Error processing callback for node 69bf85adf1004e82292e44b9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43294 - "GET / HTTP/1.0" 200 OK 2026-03-24 06:02:08,255 - lava_callback - INFO - Received callback for node 69bf85d2f1004e82292e44c5 INFO: 172.21.0.1:43296 - "POST /node/69bf85d2f1004e82292e44c5 HTTP/1.0" 202 Accepted 2026-03-24 06:02:08,261 - lava_callback - INFO - Starting async job submit for node 69bf85d2f1004e82292e44c5 2026-03-24 06:02:08,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:02:08,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf85d2f1004e82292e44c5 HTTP/1.1" 200 None 2026-03-24 06:02:08,277 - lava_callback - DEBUG - Processing node 69bf85d2f1004e82292e44c5: baseline-arm-clabbe 2026-03-24 06:02:08,278 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf85d2f1004e82292e44c5 2026-03-24 06:02:08,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:02:08,292 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf85d2f1004e82292e44c5 HTTP/1.1" 200 None 2026-03-24 06:02:08,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:02:08,307 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21a117e5cb936602bc16 HTTP/1.1" 200 None 2026-03-24 06:02:08,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:02:08,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf85d2f1004e82292e44c5 HTTP/1.1" 200 None 2026-03-24 06:02:08,348 - lava_callback - ERROR - Error processing callback for node 69bf85d2f1004e82292e44c5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39640 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46540 - "GET / HTTP/1.0" 200 OK 2026-03-24 06:02:48,332 - lava_callback - INFO - Received callback for node 69bf85f6f1004e82292e44cf INFO: 172.21.0.1:46552 - "POST /node/69bf85f6f1004e82292e44cf HTTP/1.0" 202 Accepted 2026-03-24 06:02:48,337 - lava_callback - INFO - Starting async job submit for node 69bf85f6f1004e82292e44cf 2026-03-24 06:02:48,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:02:48,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf85f6f1004e82292e44cf HTTP/1.1" 200 None 2026-03-24 06:02:48,354 - lava_callback - DEBUG - Processing node 69bf85f6f1004e82292e44cf: baseline-arm-clabbe 2026-03-24 06:02:48,354 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf85f6f1004e82292e44cf 2026-03-24 06:02:48,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:02:48,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf85f6f1004e82292e44cf HTTP/1.1" 200 None 2026-03-24 06:02:48,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:02:48,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21a117e5cb936602bc16 HTTP/1.1" 200 None 2026-03-24 06:02:48,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:02:48,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf85f6f1004e82292e44cf HTTP/1.1" 200 None 2026-03-24 06:02:48,419 - lava_callback - ERROR - Error processing callback for node 69bf85f6f1004e82292e44cf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49478 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50566 - "GET / HTTP/1.0" 200 OK 2026-03-24 06:03:28,309 - lava_callback - INFO - Received callback for node 69bf861af1004e82292e44dc INFO: 172.21.0.1:50572 - "POST /node/69bf861af1004e82292e44dc HTTP/1.0" 202 Accepted 2026-03-24 06:03:28,314 - lava_callback - INFO - Starting async job submit for node 69bf861af1004e82292e44dc 2026-03-24 06:03:28,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:03:28,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf861af1004e82292e44dc HTTP/1.1" 200 None 2026-03-24 06:03:28,330 - lava_callback - DEBUG - Processing node 69bf861af1004e82292e44dc: baseline-arm-clabbe 2026-03-24 06:03:28,331 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf861af1004e82292e44dc 2026-03-24 06:03:28,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:03:28,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf861af1004e82292e44dc HTTP/1.1" 200 None 2026-03-24 06:03:28,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:03:28,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21a117e5cb936602bc16 HTTP/1.1" 200 None 2026-03-24 06:03:28,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:03:28,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf861af1004e82292e44dc HTTP/1.1" 200 None 2026-03-24 06:03:28,388 - lava_callback - ERROR - Error processing callback for node 69bf861af1004e82292e44dc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56474 - "GET / HTTP/1.0" 200 OK 2026-03-24 06:03:48,374 - lava_callback - INFO - Received callback for node 69bf863ef1004e82292e44e5 INFO: 172.21.0.1:56482 - "POST /node/69bf863ef1004e82292e44e5 HTTP/1.0" 202 Accepted 2026-03-24 06:03:48,380 - lava_callback - INFO - Starting async job submit for node 69bf863ef1004e82292e44e5 2026-03-24 06:03:48,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:03:48,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf863ef1004e82292e44e5 HTTP/1.1" 200 None 2026-03-24 06:03:48,395 - lava_callback - DEBUG - Processing node 69bf863ef1004e82292e44e5: baseline-arm-clabbe 2026-03-24 06:03:48,396 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf863ef1004e82292e44e5 2026-03-24 06:03:48,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:03:48,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf863ef1004e82292e44e5 HTTP/1.1" 200 None 2026-03-24 06:03:48,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:03:48,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21a117e5cb936602bc16 HTTP/1.1" 200 None 2026-03-24 06:03:48,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:03:48,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf863ef1004e82292e44e5 HTTP/1.1" 200 None 2026-03-24 06:03:48,453 - lava_callback - ERROR - Error processing callback for node 69bf863ef1004e82292e44e5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38574 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52110 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54214 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35750 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42098 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44040 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54042 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45956 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43738 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45056 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53072 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56644 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37432 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47192 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56234 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37122 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44268 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40064 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52716 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34794 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44382 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57324 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33938 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40014 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45906 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60626 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37120 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47508 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45794 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59636 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51634 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40594 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42148 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57100 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33600 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54398 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55310 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57194 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54822 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38886 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39710 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41360 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48646 - "GET / HTTP/1.0" 200 OK 2026-03-24 06:45:50,381 - lava_callback - INFO - Received callback for node 69bf9013f1004e82292e4542 INFO: 172.21.0.1:53912 - "POST /node/69bf9013f1004e82292e4542 HTTP/1.0" 202 Accepted 2026-03-24 06:45:50,387 - lava_callback - INFO - Starting async job submit for node 69bf9013f1004e82292e4542 2026-03-24 06:45:50,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:45:50,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf9013f1004e82292e4542 HTTP/1.1" 200 None 2026-03-24 06:45:50,403 - lava_callback - DEBUG - Processing node 69bf9013f1004e82292e4542: baseline-arm-clabbe 2026-03-24 06:45:50,403 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf9013f1004e82292e4542 2026-03-24 06:45:50,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:45:50,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf9013f1004e82292e4542 HTTP/1.1" 200 None 2026-03-24 06:45:50,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:45:50,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-24 06:45:50,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:45:50,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf9013f1004e82292e4542 HTTP/1.1" 200 None 2026-03-24 06:45:50,460 - lava_callback - ERROR - Error processing callback for node 69bf9013f1004e82292e4542: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46952 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33450 - "GET / HTTP/1.0" 200 OK 2026-03-24 06:46:30,201 - lava_callback - INFO - Received callback for node 69bf9038f1004e82292e4549 INFO: 172.21.0.1:57690 - "POST /node/69bf9038f1004e82292e4549 HTTP/1.0" 202 Accepted 2026-03-24 06:46:30,206 - lava_callback - INFO - Starting async job submit for node 69bf9038f1004e82292e4549 2026-03-24 06:46:30,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:46:30,221 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf9038f1004e82292e4549 HTTP/1.1" 200 None 2026-03-24 06:46:30,222 - lava_callback - DEBUG - Processing node 69bf9038f1004e82292e4549: baseline-arm-clabbe 2026-03-24 06:46:30,222 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf9038f1004e82292e4549 2026-03-24 06:46:30,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:46:30,236 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf9038f1004e82292e4549 HTTP/1.1" 200 None 2026-03-24 06:46:30,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:46:30,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-24 06:46:30,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:46:30,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf9038f1004e82292e4549 HTTP/1.1" 200 None 2026-03-24 06:46:30,295 - lava_callback - ERROR - Error processing callback for node 69bf9038f1004e82292e4549: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40520 - "GET / HTTP/1.0" 200 OK 2026-03-24 06:47:10,301 - lava_callback - INFO - Received callback for node 69bf905cf1004e82292e4553 INFO: 172.21.0.1:45628 - "POST /node/69bf905cf1004e82292e4553 HTTP/1.0" 202 Accepted 2026-03-24 06:47:10,307 - lava_callback - INFO - Starting async job submit for node 69bf905cf1004e82292e4553 2026-03-24 06:47:10,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:47:10,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf905cf1004e82292e4553 HTTP/1.1" 200 None 2026-03-24 06:47:10,322 - lava_callback - DEBUG - Processing node 69bf905cf1004e82292e4553: baseline-arm-clabbe 2026-03-24 06:47:10,322 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf905cf1004e82292e4553 2026-03-24 06:47:10,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:47:10,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf905cf1004e82292e4553 HTTP/1.1" 200 None 2026-03-24 06:47:10,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:47:10,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-24 06:47:10,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:47:10,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf905cf1004e82292e4553 HTTP/1.1" 200 None 2026-03-24 06:47:10,412 - lava_callback - ERROR - Error processing callback for node 69bf905cf1004e82292e4553: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45652 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51906 - "GET / HTTP/1.0" 200 OK 2026-03-24 06:47:50,279 - lava_callback - INFO - Received callback for node 69bf9081f1004e82292e455f INFO: 172.21.0.1:35208 - "POST /node/69bf9081f1004e82292e455f HTTP/1.0" 202 Accepted 2026-03-24 06:47:50,285 - lava_callback - INFO - Starting async job submit for node 69bf9081f1004e82292e455f 2026-03-24 06:47:50,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:47:50,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf9081f1004e82292e455f HTTP/1.1" 200 None 2026-03-24 06:47:50,302 - lava_callback - DEBUG - Processing node 69bf9081f1004e82292e455f: baseline-arm-clabbe 2026-03-24 06:47:50,302 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf9081f1004e82292e455f 2026-03-24 06:47:50,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:47:50,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf9081f1004e82292e455f HTTP/1.1" 200 None 2026-03-24 06:47:50,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:47:50,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-24 06:47:50,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:47:50,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf9081f1004e82292e455f HTTP/1.1" 200 None 2026-03-24 06:47:50,354 - lava_callback - ERROR - Error processing callback for node 69bf9081f1004e82292e455f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33578 - "GET / HTTP/1.0" 200 OK 2026-03-24 06:48:10,337 - lava_callback - INFO - Received callback for node 69bf90a6f1004e82292e456e INFO: 172.21.0.1:51444 - "POST /node/69bf90a6f1004e82292e456e HTTP/1.0" 202 Accepted 2026-03-24 06:48:10,342 - lava_callback - INFO - Starting async job submit for node 69bf90a6f1004e82292e456e 2026-03-24 06:48:10,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:48:10,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf90a6f1004e82292e456e HTTP/1.1" 200 None 2026-03-24 06:48:10,365 - lava_callback - DEBUG - Processing node 69bf90a6f1004e82292e456e: baseline-arm-clabbe 2026-03-24 06:48:10,366 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf90a6f1004e82292e456e 2026-03-24 06:48:10,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:48:10,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf90a6f1004e82292e456e HTTP/1.1" 200 None 2026-03-24 06:48:10,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:48:10,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-24 06:48:10,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:48:10,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf90a6f1004e82292e456e HTTP/1.1" 200 None 2026-03-24 06:48:10,425 - lava_callback - ERROR - Error processing callback for node 69bf90a6f1004e82292e456e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51456 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58200 - "GET / HTTP/1.0" 200 OK 2026-03-24 06:48:50,307 - lava_callback - INFO - Received callback for node 69bf90caf1004e82292e4579 INFO: 172.21.0.1:56126 - "POST /node/69bf90caf1004e82292e4579 HTTP/1.0" 202 Accepted 2026-03-24 06:48:50,312 - lava_callback - INFO - Starting async job submit for node 69bf90caf1004e82292e4579 2026-03-24 06:48:50,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:48:50,327 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf90caf1004e82292e4579 HTTP/1.1" 200 None 2026-03-24 06:48:50,328 - lava_callback - DEBUG - Processing node 69bf90caf1004e82292e4579: baseline-arm-clabbe 2026-03-24 06:48:50,328 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf90caf1004e82292e4579 2026-03-24 06:48:50,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:48:50,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf90caf1004e82292e4579 HTTP/1.1" 200 None 2026-03-24 06:48:50,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:48:50,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92d5b4744551b27f9fbe HTTP/1.1" 200 None 2026-03-24 06:48:50,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:48:50,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf90caf1004e82292e4579 HTTP/1.1" 200 None 2026-03-24 06:48:50,402 - lava_callback - ERROR - Error processing callback for node 69bf90caf1004e82292e4579: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60422 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60380 - "GET / HTTP/1.0" 200 OK 2026-03-24 06:49:30,372 - lava_callback - INFO - Received callback for node 69bf90eef1004e82292e4585 INFO: 172.21.0.1:51752 - "POST /node/69bf90eef1004e82292e4585 HTTP/1.0" 202 Accepted 2026-03-24 06:49:30,378 - lava_callback - INFO - Starting async job submit for node 69bf90eef1004e82292e4585 2026-03-24 06:49:30,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:49:30,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf90eef1004e82292e4585 HTTP/1.1" 200 None 2026-03-24 06:49:30,393 - lava_callback - DEBUG - Processing node 69bf90eef1004e82292e4585: baseline-arm-clabbe 2026-03-24 06:49:30,394 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf90eef1004e82292e4585 2026-03-24 06:49:30,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:49:30,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf90eef1004e82292e4585 HTTP/1.1" 200 None 2026-03-24 06:49:30,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:49:30,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92d5b4744551b27f9fbe HTTP/1.1" 200 None 2026-03-24 06:49:30,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:49:30,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf90eef1004e82292e4585 HTTP/1.1" 200 None 2026-03-24 06:49:30,484 - lava_callback - ERROR - Error processing callback for node 69bf90eef1004e82292e4585: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36538 - "GET / HTTP/1.0" 200 OK 2026-03-24 06:50:10,403 - lava_callback - INFO - Received callback for node 69bf9113f1004e82292e4592 INFO: 172.21.0.1:45344 - "POST /node/69bf9113f1004e82292e4592 HTTP/1.0" 202 Accepted 2026-03-24 06:50:10,409 - lava_callback - INFO - Starting async job submit for node 69bf9113f1004e82292e4592 2026-03-24 06:50:10,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:50:10,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf9113f1004e82292e4592 HTTP/1.1" 200 None 2026-03-24 06:50:10,425 - lava_callback - DEBUG - Processing node 69bf9113f1004e82292e4592: baseline-arm-clabbe 2026-03-24 06:50:10,426 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf9113f1004e82292e4592 2026-03-24 06:50:10,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:50:10,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf9113f1004e82292e4592 HTTP/1.1" 200 None 2026-03-24 06:50:10,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:50:10,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92d5b4744551b27f9fbe HTTP/1.1" 200 None 2026-03-24 06:50:10,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:50:10,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf9113f1004e82292e4592 HTTP/1.1" 200 None 2026-03-24 06:50:10,485 - lava_callback - ERROR - Error processing callback for node 69bf9113f1004e82292e4592: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45358 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43468 - "GET / HTTP/1.0" 200 OK 2026-03-24 06:50:50,453 - lava_callback - INFO - Received callback for node 69bf9139f1004e82292e459f INFO: 172.21.0.1:43004 - "POST /node/69bf9139f1004e82292e459f HTTP/1.0" 202 Accepted 2026-03-24 06:50:50,458 - lava_callback - INFO - Starting async job submit for node 69bf9139f1004e82292e459f 2026-03-24 06:50:50,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:50:50,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf9139f1004e82292e459f HTTP/1.1" 200 None 2026-03-24 06:50:50,475 - lava_callback - DEBUG - Processing node 69bf9139f1004e82292e459f: baseline-arm-clabbe 2026-03-24 06:50:50,475 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf9139f1004e82292e459f 2026-03-24 06:50:50,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:50:50,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf9139f1004e82292e459f HTTP/1.1" 200 None 2026-03-24 06:50:50,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:50:50,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92d5b4744551b27f9fbe HTTP/1.1" 200 None 2026-03-24 06:50:50,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:50:50,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf9139f1004e82292e459f HTTP/1.1" 200 None 2026-03-24 06:50:50,533 - lava_callback - ERROR - Error processing callback for node 69bf9139f1004e82292e459f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55078 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46200 - "GET / HTTP/1.0" 200 OK 2026-03-24 06:51:30,463 - lava_callback - INFO - Received callback for node 69bf915ef1004e82292e45ae INFO: 172.21.0.1:44872 - "POST /node/69bf915ef1004e82292e45ae HTTP/1.0" 202 Accepted 2026-03-24 06:51:30,470 - lava_callback - INFO - Starting async job submit for node 69bf915ef1004e82292e45ae 2026-03-24 06:51:30,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:51:30,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf915ef1004e82292e45ae HTTP/1.1" 200 None 2026-03-24 06:51:30,487 - lava_callback - DEBUG - Processing node 69bf915ef1004e82292e45ae: baseline-arm-clabbe 2026-03-24 06:51:30,487 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf915ef1004e82292e45ae 2026-03-24 06:51:30,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:51:30,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf915ef1004e82292e45ae HTTP/1.1" 200 None 2026-03-24 06:51:30,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:51:30,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92d5b4744551b27f9fbe HTTP/1.1" 200 None 2026-03-24 06:51:30,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:51:30,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf915ef1004e82292e45ae HTTP/1.1" 200 None 2026-03-24 06:51:30,547 - lava_callback - ERROR - Error processing callback for node 69bf915ef1004e82292e45ae: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39264 - "GET / HTTP/1.0" 200 OK 2026-03-24 06:51:50,517 - lava_callback - INFO - Received callback for node 69bf9182f1004e82292e45b6 INFO: 172.21.0.1:37552 - "POST /node/69bf9182f1004e82292e45b6 HTTP/1.0" 202 Accepted 2026-03-24 06:51:50,522 - lava_callback - INFO - Starting async job submit for node 69bf9182f1004e82292e45b6 2026-03-24 06:51:50,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:51:50,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf9182f1004e82292e45b6 HTTP/1.1" 200 None 2026-03-24 06:51:50,540 - lava_callback - DEBUG - Processing node 69bf9182f1004e82292e45b6: baseline-arm-clabbe 2026-03-24 06:51:50,541 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf9182f1004e82292e45b6 2026-03-24 06:51:50,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:51:50,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf9182f1004e82292e45b6 HTTP/1.1" 200 None 2026-03-24 06:51:50,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:51:50,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92d5b4744551b27f9fbe HTTP/1.1" 200 None 2026-03-24 06:51:50,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:51:50,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf9182f1004e82292e45b6 HTTP/1.1" 200 None 2026-03-24 06:51:50,600 - lava_callback - ERROR - Error processing callback for node 69bf9182f1004e82292e45b6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53602 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42262 - "GET / HTTP/1.0" 200 OK 2026-03-24 06:52:30,426 - lava_callback - INFO - Received callback for node 69bf91a5f1004e82292e45c4 INFO: 172.21.0.1:51826 - "POST /node/69bf91a5f1004e82292e45c4 HTTP/1.0" 202 Accepted 2026-03-24 06:52:30,432 - lava_callback - INFO - Starting async job submit for node 69bf91a5f1004e82292e45c4 2026-03-24 06:52:30,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:52:30,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf91a5f1004e82292e45c4 HTTP/1.1" 200 None 2026-03-24 06:52:30,448 - lava_callback - DEBUG - Processing node 69bf91a5f1004e82292e45c4: baseline-arm-clabbe 2026-03-24 06:52:30,448 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf91a5f1004e82292e45c4 2026-03-24 06:52:30,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:52:30,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf91a5f1004e82292e45c4 HTTP/1.1" 200 None 2026-03-24 06:52:30,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:52:30,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92d5b4744551b27f9fbe HTTP/1.1" 200 None 2026-03-24 06:52:30,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:52:30,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf91a5f1004e82292e45c4 HTTP/1.1" 200 None 2026-03-24 06:52:30,507 - lava_callback - ERROR - Error processing callback for node 69bf91a5f1004e82292e45c4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52008 - "GET / HTTP/1.0" 200 OK 2026-03-24 06:53:10,566 - lava_callback - INFO - Received callback for node 69bf91c9f1004e82292e45d0 INFO: 172.21.0.1:41444 - "POST /node/69bf91c9f1004e82292e45d0 HTTP/1.0" 202 Accepted 2026-03-24 06:53:10,573 - lava_callback - INFO - Starting async job submit for node 69bf91c9f1004e82292e45d0 2026-03-24 06:53:10,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:53:10,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf91c9f1004e82292e45d0 HTTP/1.1" 200 None 2026-03-24 06:53:10,588 - lava_callback - DEBUG - Processing node 69bf91c9f1004e82292e45d0: baseline-arm-clabbe 2026-03-24 06:53:10,589 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf91c9f1004e82292e45d0 2026-03-24 06:53:10,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:53:10,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf91c9f1004e82292e45d0 HTTP/1.1" 200 None 2026-03-24 06:53:10,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:53:10,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92d5b4744551b27f9fbe HTTP/1.1" 200 None 2026-03-24 06:53:10,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:53:10,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf91c9f1004e82292e45d0 HTTP/1.1" 200 None 2026-03-24 06:53:10,658 - lava_callback - ERROR - Error processing callback for node 69bf91c9f1004e82292e45d0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41456 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34394 - "GET / HTTP/1.0" 200 OK 2026-03-24 06:53:50,476 - lava_callback - INFO - Received callback for node 69bf91edf1004e82292e45da INFO: 172.21.0.1:39386 - "POST /node/69bf91edf1004e82292e45da HTTP/1.0" 202 Accepted 2026-03-24 06:53:50,482 - lava_callback - INFO - Starting async job submit for node 69bf91edf1004e82292e45da 2026-03-24 06:53:50,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:53:50,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf91edf1004e82292e45da HTTP/1.1" 200 None 2026-03-24 06:53:50,497 - lava_callback - DEBUG - Processing node 69bf91edf1004e82292e45da: baseline-arm-clabbe 2026-03-24 06:53:50,498 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf91edf1004e82292e45da 2026-03-24 06:53:50,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:53:50,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf91edf1004e82292e45da HTTP/1.1" 200 None 2026-03-24 06:53:50,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:53:50,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92d5b4744551b27f9fbe HTTP/1.1" 200 None 2026-03-24 06:53:50,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:53:50,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf91edf1004e82292e45da HTTP/1.1" 200 None 2026-03-24 06:53:50,587 - lava_callback - ERROR - Error processing callback for node 69bf91edf1004e82292e45da: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40876 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43328 - "GET / HTTP/1.0" 200 OK 2026-03-24 06:54:30,587 - lava_callback - INFO - Received callback for node 69bf9211f1004e82292e45e8 INFO: 172.21.0.1:39982 - "POST /node/69bf9211f1004e82292e45e8 HTTP/1.0" 202 Accepted 2026-03-24 06:54:30,593 - lava_callback - INFO - Starting async job submit for node 69bf9211f1004e82292e45e8 2026-03-24 06:54:30,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:54:30,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf9211f1004e82292e45e8 HTTP/1.1" 200 None 2026-03-24 06:54:30,615 - lava_callback - DEBUG - Processing node 69bf9211f1004e82292e45e8: baseline-arm-clabbe 2026-03-24 06:54:30,616 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf9211f1004e82292e45e8 2026-03-24 06:54:30,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:54:30,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf9211f1004e82292e45e8 HTTP/1.1" 200 None 2026-03-24 06:54:30,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:54:30,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92d5b4744551b27f9fbe HTTP/1.1" 200 None 2026-03-24 06:54:30,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:54:30,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf9211f1004e82292e45e8 HTTP/1.1" 200 None 2026-03-24 06:54:30,679 - lava_callback - ERROR - Error processing callback for node 69bf9211f1004e82292e45e8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48162 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47214 - "GET / HTTP/1.0" 200 OK 2026-03-24 06:55:53,361 - lava_callback - INFO - Received callback for node 69bf926af1004e82292e45f2 INFO: 172.21.0.1:49128 - "POST /node/69bf926af1004e82292e45f2 HTTP/1.0" 202 Accepted 2026-03-24 06:55:53,367 - lava_callback - INFO - Starting async job submit for node 69bf926af1004e82292e45f2 2026-03-24 06:55:53,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:55:53,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf926af1004e82292e45f2 HTTP/1.1" 200 None 2026-03-24 06:55:53,387 - lava_callback - DEBUG - Processing node 69bf926af1004e82292e45f2: baseline-arm-baylibre 2026-03-24 06:55:53,387 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf926af1004e82292e45f2 2026-03-24 06:55:53,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:55:53,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf926af1004e82292e45f2 HTTP/1.1" 200 None 2026-03-24 06:55:53,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:55:53,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef76d5731cbd74145fb9 HTTP/1.1" 200 None 2026-03-24 06:55:53,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:55:53,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf926af1004e82292e45f2 HTTP/1.1" 200 None 2026-03-24 06:55:53,447 - lava_callback - ERROR - Error processing callback for node 69bf926af1004e82292e45f2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44446 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38602 - "GET / HTTP/1.0" 200 OK 2026-03-24 06:56:33,124 - lava_callback - INFO - Received callback for node 69bf9293f1004e82292e45fc INFO: 172.21.0.1:47560 - "POST /node/69bf9293f1004e82292e45fc HTTP/1.0" 202 Accepted 2026-03-24 06:56:33,130 - lava_callback - INFO - Starting async job submit for node 69bf9293f1004e82292e45fc 2026-03-24 06:56:33,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:56:33,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf9293f1004e82292e45fc HTTP/1.1" 200 None 2026-03-24 06:56:33,151 - lava_callback - DEBUG - Processing node 69bf9293f1004e82292e45fc: baseline-arm-baylibre 2026-03-24 06:56:33,151 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf9293f1004e82292e45fc 2026-03-24 06:56:33,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:56:33,166 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf9293f1004e82292e45fc HTTP/1.1" 200 None 2026-03-24 06:56:33,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:56:33,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef76d5731cbd74145fb9 HTTP/1.1" 200 None 2026-03-24 06:56:33,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:56:33,210 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf9293f1004e82292e45fc HTTP/1.1" 200 None 2026-03-24 06:56:33,212 - lava_callback - ERROR - Error processing callback for node 69bf9293f1004e82292e45fc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58096 - "GET / HTTP/1.0" 200 OK 2026-03-24 06:57:13,235 - lava_callback - INFO - Received callback for node 69bf92bbf1004e82292e4608 INFO: 172.21.0.1:42600 - "POST /node/69bf92bbf1004e82292e4608 HTTP/1.0" 202 Accepted 2026-03-24 06:57:13,240 - lava_callback - INFO - Starting async job submit for node 69bf92bbf1004e82292e4608 2026-03-24 06:57:13,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:57:13,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf92bbf1004e82292e4608 HTTP/1.1" 200 None 2026-03-24 06:57:13,257 - lava_callback - DEBUG - Processing node 69bf92bbf1004e82292e4608: baseline-arm-baylibre 2026-03-24 06:57:13,258 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf92bbf1004e82292e4608 2026-03-24 06:57:13,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:57:13,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf92bbf1004e82292e4608 HTTP/1.1" 200 None 2026-03-24 06:57:13,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:57:13,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef76d5731cbd74145fb9 HTTP/1.1" 200 None 2026-03-24 06:57:13,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:57:13,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf92bbf1004e82292e4608 HTTP/1.1" 200 None 2026-03-24 06:57:13,586 - lava_callback - ERROR - Error processing callback for node 69bf92bbf1004e82292e4608: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42624 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57348 - "GET / HTTP/1.0" 200 OK 2026-03-24 06:57:53,224 - lava_callback - INFO - Received callback for node 69bf92e4f1004e82292e4615 INFO: 172.21.0.1:55572 - "POST /node/69bf92e4f1004e82292e4615 HTTP/1.0" 202 Accepted 2026-03-24 06:57:53,230 - lava_callback - INFO - Starting async job submit for node 69bf92e4f1004e82292e4615 2026-03-24 06:57:53,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:57:53,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf92e4f1004e82292e4615 HTTP/1.1" 200 None 2026-03-24 06:57:53,247 - lava_callback - DEBUG - Processing node 69bf92e4f1004e82292e4615: baseline-arm-baylibre 2026-03-24 06:57:53,247 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf92e4f1004e82292e4615 2026-03-24 06:57:53,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:57:53,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf92e4f1004e82292e4615 HTTP/1.1" 200 None 2026-03-24 06:57:53,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:57:53,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef76d5731cbd74145fb9 HTTP/1.1" 200 None 2026-03-24 06:57:53,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:57:53,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf92e4f1004e82292e4615 HTTP/1.1" 200 None 2026-03-24 06:57:53,303 - lava_callback - ERROR - Error processing callback for node 69bf92e4f1004e82292e4615: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43388 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46840 - "GET / HTTP/1.0" 200 OK 2026-03-24 06:58:33,291 - lava_callback - INFO - Received callback for node 69bf930df1004e82292e4620 INFO: 172.21.0.1:46798 - "POST /node/69bf930df1004e82292e4620 HTTP/1.0" 202 Accepted 2026-03-24 06:58:33,305 - lava_callback - INFO - Starting async job submit for node 69bf930df1004e82292e4620 2026-03-24 06:58:33,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:58:33,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf930df1004e82292e4620 HTTP/1.1" 200 None 2026-03-24 06:58:33,336 - lava_callback - DEBUG - Processing node 69bf930df1004e82292e4620: baseline-arm-baylibre 2026-03-24 06:58:33,337 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf930df1004e82292e4620 2026-03-24 06:58:33,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:58:33,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf930df1004e82292e4620 HTTP/1.1" 200 None 2026-03-24 06:58:33,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:58:33,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef76d5731cbd74145fb9 HTTP/1.1" 200 None 2026-03-24 06:58:33,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:58:33,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf930df1004e82292e4620 HTTP/1.1" 200 None 2026-03-24 06:58:33,413 - lava_callback - ERROR - Error processing callback for node 69bf930df1004e82292e4620: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33760 - "GET / HTTP/1.0" 200 OK 2026-03-24 06:59:13,294 - lava_callback - INFO - Received callback for node 69bf9336f1004e82292e462c INFO: 172.21.0.1:55330 - "POST /node/69bf9336f1004e82292e462c HTTP/1.0" 202 Accepted 2026-03-24 06:59:13,300 - lava_callback - INFO - Starting async job submit for node 69bf9336f1004e82292e462c 2026-03-24 06:59:13,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:59:13,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf9336f1004e82292e462c HTTP/1.1" 200 None 2026-03-24 06:59:13,317 - lava_callback - DEBUG - Processing node 69bf9336f1004e82292e462c: baseline-arm-baylibre 2026-03-24 06:59:13,317 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf9336f1004e82292e462c 2026-03-24 06:59:13,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:59:13,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf9336f1004e82292e462c HTTP/1.1" 200 None 2026-03-24 06:59:13,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:59:13,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef76d5731cbd74145fb9 HTTP/1.1" 200 None 2026-03-24 06:59:13,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:59:13,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf9336f1004e82292e462c HTTP/1.1" 200 None 2026-03-24 06:59:13,370 - lava_callback - ERROR - Error processing callback for node 69bf9336f1004e82292e462c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55344 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56820 - "GET / HTTP/1.0" 200 OK 2026-03-24 06:59:53,405 - lava_callback - INFO - Received callback for node 69bf9355f1004e82292e463a INFO: 172.21.0.1:45532 - "POST /node/69bf9355f1004e82292e463a HTTP/1.0" 202 Accepted 2026-03-24 06:59:53,410 - lava_callback - INFO - Starting async job submit for node 69bf9355f1004e82292e463a 2026-03-24 06:59:53,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:59:53,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf9355f1004e82292e463a HTTP/1.1" 200 None 2026-03-24 06:59:53,429 - lava_callback - DEBUG - Processing node 69bf9355f1004e82292e463a: baseline-arm-baylibre 2026-03-24 06:59:53,429 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf9355f1004e82292e463a 2026-03-24 06:59:53,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:59:53,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf9355f1004e82292e463a HTTP/1.1" 200 None 2026-03-24 06:59:53,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:59:53,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef76d5731cbd74145fb9 HTTP/1.1" 200 None 2026-03-24 06:59:53,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 06:59:53,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf9355f1004e82292e463a HTTP/1.1" 200 None 2026-03-24 06:59:53,514 - lava_callback - ERROR - Error processing callback for node 69bf9355f1004e82292e463a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36534 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47056 - "GET / HTTP/1.0" 200 OK 2026-03-24 07:00:53,376 - lava_callback - INFO - Received callback for node 69bf939cf1004e82292e464f INFO: 172.21.0.1:52660 - "POST /node/69bf939cf1004e82292e464f HTTP/1.0" 202 Accepted 2026-03-24 07:00:53,382 - lava_callback - INFO - Starting async job submit for node 69bf939cf1004e82292e464f 2026-03-24 07:00:53,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:00:53,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf939cf1004e82292e464f HTTP/1.1" 200 None 2026-03-24 07:00:53,484 - lava_callback - DEBUG - Processing node 69bf939cf1004e82292e464f: baseline-arm-baylibre 2026-03-24 07:00:53,484 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf939cf1004e82292e464f 2026-03-24 07:00:53,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:00:53,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf939cf1004e82292e464f HTTP/1.1" 200 None 2026-03-24 07:00:53,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:00:53,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef76d5731cbd74145fb9 HTTP/1.1" 200 None 2026-03-24 07:00:53,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:00:53,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf939cf1004e82292e464f HTTP/1.1" 200 None 2026-03-24 07:00:53,556 - lava_callback - ERROR - Error processing callback for node 69bf939cf1004e82292e464f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33608 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35838 - "GET / HTTP/1.0" 200 OK 2026-03-24 07:01:33,465 - lava_callback - INFO - Received callback for node 69bf93bff1004e82292e465b INFO: 172.21.0.1:51618 - "POST /node/69bf93bff1004e82292e465b HTTP/1.0" 202 Accepted 2026-03-24 07:01:33,470 - lava_callback - INFO - Starting async job submit for node 69bf93bff1004e82292e465b 2026-03-24 07:01:33,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:01:33,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf93bff1004e82292e465b HTTP/1.1" 200 None 2026-03-24 07:01:33,486 - lava_callback - DEBUG - Processing node 69bf93bff1004e82292e465b: baseline-arm-baylibre 2026-03-24 07:01:33,486 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf93bff1004e82292e465b 2026-03-24 07:01:33,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:01:33,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf93bff1004e82292e465b HTTP/1.1" 200 None 2026-03-24 07:01:33,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:01:33,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef76d5731cbd74145fb9 HTTP/1.1" 200 None 2026-03-24 07:01:33,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:01:33,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf93bff1004e82292e465b HTTP/1.1" 200 None 2026-03-24 07:01:33,556 - lava_callback - ERROR - Error processing callback for node 69bf93bff1004e82292e465b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53238 - "GET / HTTP/1.0" 200 OK 2026-03-24 07:02:13,463 - lava_callback - INFO - Received callback for node 69bf93e3f1004e82292e4669 INFO: 172.21.0.1:44010 - "POST /node/69bf93e3f1004e82292e4669 HTTP/1.0" 202 Accepted 2026-03-24 07:02:13,469 - lava_callback - INFO - Starting async job submit for node 69bf93e3f1004e82292e4669 2026-03-24 07:02:13,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:02:13,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf93e3f1004e82292e4669 HTTP/1.1" 200 None 2026-03-24 07:02:13,485 - lava_callback - DEBUG - Processing node 69bf93e3f1004e82292e4669: baseline-arm-baylibre 2026-03-24 07:02:13,485 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf93e3f1004e82292e4669 2026-03-24 07:02:13,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:02:13,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf93e3f1004e82292e4669 HTTP/1.1" 200 None 2026-03-24 07:02:13,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:02:13,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef76d5731cbd74145fb9 HTTP/1.1" 200 None 2026-03-24 07:02:13,517 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:02:13,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf93e3f1004e82292e4669 HTTP/1.1" 200 None 2026-03-24 07:02:13,544 - lava_callback - ERROR - Error processing callback for node 69bf93e3f1004e82292e4669: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44018 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56872 - "GET / HTTP/1.0" 200 OK 2026-03-24 07:02:33,502 - lava_callback - INFO - Received callback for node 69bf9406f1004e82292e467a INFO: 172.21.0.1:37564 - "POST /node/69bf9406f1004e82292e467a HTTP/1.0" 202 Accepted 2026-03-24 07:02:33,508 - lava_callback - INFO - Starting async job submit for node 69bf9406f1004e82292e467a 2026-03-24 07:02:33,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:02:33,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf9406f1004e82292e467a HTTP/1.1" 200 None 2026-03-24 07:02:33,524 - lava_callback - DEBUG - Processing node 69bf9406f1004e82292e467a: baseline-arm-baylibre 2026-03-24 07:02:33,524 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf9406f1004e82292e467a 2026-03-24 07:02:33,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:02:33,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf9406f1004e82292e467a HTTP/1.1" 200 None 2026-03-24 07:02:33,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:02:33,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef76d5731cbd74145fb9 HTTP/1.1" 200 None 2026-03-24 07:02:33,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:02:33,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf9406f1004e82292e467a HTTP/1.1" 200 None 2026-03-24 07:02:33,668 - lava_callback - ERROR - Error processing callback for node 69bf9406f1004e82292e467a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48896 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53170 - "GET / HTTP/1.0" 200 OK 2026-03-24 07:03:30,939 - lava_callback - INFO - Received callback for node 69bf942df1004e82292e468b INFO: 172.21.0.1:35428 - "POST /node/69bf942df1004e82292e468b HTTP/1.0" 202 Accepted 2026-03-24 07:03:30,944 - lava_callback - INFO - Starting async job submit for node 69bf942df1004e82292e468b 2026-03-24 07:03:30,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:03:30,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf942df1004e82292e468b HTTP/1.1" 200 None 2026-03-24 07:03:30,960 - lava_callback - DEBUG - Processing node 69bf942df1004e82292e468b: baseline-arm-clabbe 2026-03-24 07:03:30,960 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf942df1004e82292e468b 2026-03-24 07:03:30,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:03:30,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf942df1004e82292e468b HTTP/1.1" 200 None 2026-03-24 07:03:30,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:03:31,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef76d5731cbd74145fb9 HTTP/1.1" 200 None 2026-03-24 07:03:31,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:03:31,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf942df1004e82292e468b HTTP/1.1" 200 None 2026-03-24 07:03:31,056 - lava_callback - ERROR - Error processing callback for node 69bf942df1004e82292e468b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35998 - "GET / HTTP/1.0" 200 OK 2026-03-24 07:03:53,571 - lava_callback - INFO - Received callback for node 69bf944df1004e82292e4698 INFO: 172.21.0.1:52878 - "POST /node/69bf944df1004e82292e4698 HTTP/1.0" 202 Accepted 2026-03-24 07:03:53,576 - lava_callback - INFO - Starting async job submit for node 69bf944df1004e82292e4698 2026-03-24 07:03:53,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:03:53,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf944df1004e82292e4698 HTTP/1.1" 200 None 2026-03-24 07:03:53,598 - lava_callback - DEBUG - Processing node 69bf944df1004e82292e4698: baseline-arm-baylibre 2026-03-24 07:03:53,598 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf944df1004e82292e4698 2026-03-24 07:03:53,599 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:03:53,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf944df1004e82292e4698 HTTP/1.1" 200 None 2026-03-24 07:03:53,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:03:53,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef76d5731cbd74145fb9 HTTP/1.1" 200 None 2026-03-24 07:03:53,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:03:53,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf944df1004e82292e4698 HTTP/1.1" 200 None 2026-03-24 07:03:53,657 - lava_callback - ERROR - Error processing callback for node 69bf944df1004e82292e4698: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60384 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52598 - "GET / HTTP/1.0" 200 OK 2026-03-24 07:04:31,009 - lava_callback - INFO - Received callback for node 69bf9474f1004e82292e46a8 INFO: 172.21.0.1:53354 - "POST /node/69bf9474f1004e82292e46a8 HTTP/1.0" 202 Accepted 2026-03-24 07:04:31,014 - lava_callback - INFO - Starting async job submit for node 69bf9474f1004e82292e46a8 2026-03-24 07:04:31,015 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:04:31,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf9474f1004e82292e46a8 HTTP/1.1" 200 None 2026-03-24 07:04:31,031 - lava_callback - DEBUG - Processing node 69bf9474f1004e82292e46a8: baseline-arm-clabbe 2026-03-24 07:04:31,031 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf9474f1004e82292e46a8 2026-03-24 07:04:31,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:04:31,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf9474f1004e82292e46a8 HTTP/1.1" 200 None 2026-03-24 07:04:31,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:04:31,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef76d5731cbd74145fb9 HTTP/1.1" 200 None 2026-03-24 07:04:31,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:04:31,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf9474f1004e82292e46a8 HTTP/1.1" 200 None 2026-03-24 07:04:31,101 - lava_callback - ERROR - Error processing callback for node 69bf9474f1004e82292e46a8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46198 - "GET / HTTP/1.0" 200 OK 2026-03-24 07:05:13,624 - lava_callback - INFO - Received callback for node 69bf9495f1004e82292e46b5 INFO: 172.21.0.1:50996 - "POST /node/69bf9495f1004e82292e46b5 HTTP/1.0" 202 Accepted 2026-03-24 07:05:13,629 - lava_callback - INFO - Starting async job submit for node 69bf9495f1004e82292e46b5 2026-03-24 07:05:13,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:05:13,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf9495f1004e82292e46b5 HTTP/1.1" 200 None 2026-03-24 07:05:13,645 - lava_callback - DEBUG - Processing node 69bf9495f1004e82292e46b5: baseline-arm-baylibre 2026-03-24 07:05:13,646 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf9495f1004e82292e46b5 2026-03-24 07:05:13,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:05:13,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf9495f1004e82292e46b5 HTTP/1.1" 200 None 2026-03-24 07:05:13,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:05:13,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef76d5731cbd74145fb9 HTTP/1.1" 200 None 2026-03-24 07:05:13,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:05:13,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf9495f1004e82292e46b5 HTTP/1.1" 200 None 2026-03-24 07:05:13,696 - lava_callback - ERROR - Error processing callback for node 69bf9495f1004e82292e46b5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51024 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56300 - "GET / HTTP/1.0" 200 OK 2026-03-24 07:05:33,627 - lava_callback - INFO - Received callback for node 69bf94b9f1004e82292e46c5 INFO: 172.21.0.1:53602 - "POST /node/69bf94b9f1004e82292e46c5 HTTP/1.0" 202 Accepted 2026-03-24 07:05:33,633 - lava_callback - INFO - Starting async job submit for node 69bf94b9f1004e82292e46c5 2026-03-24 07:05:33,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:05:33,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf94b9f1004e82292e46c5 HTTP/1.1" 200 None 2026-03-24 07:05:33,680 - lava_callback - DEBUG - Processing node 69bf94b9f1004e82292e46c5: baseline-arm-baylibre 2026-03-24 07:05:33,680 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf94b9f1004e82292e46c5 2026-03-24 07:05:33,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:05:33,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf94b9f1004e82292e46c5 HTTP/1.1" 200 None 2026-03-24 07:05:33,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:05:33,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef76d5731cbd74145fb9 HTTP/1.1" 200 None 2026-03-24 07:05:33,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:05:33,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf94b9f1004e82292e46c5 HTTP/1.1" 200 None 2026-03-24 07:05:33,744 - lava_callback - ERROR - Error processing callback for node 69bf94b9f1004e82292e46c5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55478 - "GET / HTTP/1.0" 200 OK 2026-03-24 07:06:11,035 - lava_callback - INFO - Received callback for node 69bf94def1004e82292e46d3 INFO: 172.21.0.1:53326 - "POST /node/69bf94def1004e82292e46d3 HTTP/1.0" 202 Accepted 2026-03-24 07:06:11,040 - lava_callback - INFO - Starting async job submit for node 69bf94def1004e82292e46d3 2026-03-24 07:06:11,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:06:11,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf94def1004e82292e46d3 HTTP/1.1" 200 None 2026-03-24 07:06:11,081 - lava_callback - DEBUG - Processing node 69bf94def1004e82292e46d3: baseline-arm-clabbe 2026-03-24 07:06:11,082 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf94def1004e82292e46d3 2026-03-24 07:06:11,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:06:11,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf94def1004e82292e46d3 HTTP/1.1" 200 None 2026-03-24 07:06:11,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:06:11,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef76d5731cbd74145fb9 HTTP/1.1" 200 None 2026-03-24 07:06:11,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:06:11,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf94def1004e82292e46d3 HTTP/1.1" 200 None 2026-03-24 07:06:11,153 - lava_callback - ERROR - Error processing callback for node 69bf94def1004e82292e46d3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53342 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55594 - "GET / HTTP/1.0" 200 OK 2026-03-24 07:06:51,052 - lava_callback - INFO - Received callback for node 69bf9502f1004e82292e46e6 INFO: 172.21.0.1:58290 - "POST /node/69bf9502f1004e82292e46e6 HTTP/1.0" 202 Accepted 2026-03-24 07:06:51,057 - lava_callback - INFO - Starting async job submit for node 69bf9502f1004e82292e46e6 2026-03-24 07:06:51,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:06:51,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf9502f1004e82292e46e6 HTTP/1.1" 200 None 2026-03-24 07:06:51,072 - lava_callback - DEBUG - Processing node 69bf9502f1004e82292e46e6: baseline-arm-clabbe 2026-03-24 07:06:51,073 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf9502f1004e82292e46e6 2026-03-24 07:06:51,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:06:51,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf9502f1004e82292e46e6 HTTP/1.1" 200 None 2026-03-24 07:06:51,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:06:51,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef76d5731cbd74145fb9 HTTP/1.1" 200 None 2026-03-24 07:06:51,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:06:51,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf9502f1004e82292e46e6 HTTP/1.1" 200 None 2026-03-24 07:06:51,138 - lava_callback - ERROR - Error processing callback for node 69bf9502f1004e82292e46e6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50152 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56372 - "GET / HTTP/1.0" 200 OK 2026-03-24 07:07:31,083 - lava_callback - INFO - Received callback for node 69bf9526f1004e82292e46f6 INFO: 172.21.0.1:53500 - "POST /node/69bf9526f1004e82292e46f6 HTTP/1.0" 202 Accepted 2026-03-24 07:07:31,088 - lava_callback - INFO - Starting async job submit for node 69bf9526f1004e82292e46f6 2026-03-24 07:07:31,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:07:31,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf9526f1004e82292e46f6 HTTP/1.1" 200 None 2026-03-24 07:07:31,104 - lava_callback - DEBUG - Processing node 69bf9526f1004e82292e46f6: baseline-arm-clabbe 2026-03-24 07:07:31,105 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf9526f1004e82292e46f6 2026-03-24 07:07:31,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:07:31,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf9526f1004e82292e46f6 HTTP/1.1" 200 None 2026-03-24 07:07:31,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:07:31,154 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef76d5731cbd74145fb9 HTTP/1.1" 200 None 2026-03-24 07:07:31,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:07:31,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf9526f1004e82292e46f6 HTTP/1.1" 200 None 2026-03-24 07:07:31,199 - lava_callback - ERROR - Error processing callback for node 69bf9526f1004e82292e46f6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42936 - "GET / HTTP/1.0" 200 OK 2026-03-24 07:08:11,101 - lava_callback - INFO - Received callback for node 69bf954af1004e82292e4705 INFO: 172.21.0.1:54898 - "POST /node/69bf954af1004e82292e4705 HTTP/1.0" 202 Accepted 2026-03-24 07:08:11,106 - lava_callback - INFO - Starting async job submit for node 69bf954af1004e82292e4705 2026-03-24 07:08:11,107 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:08:11,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf954af1004e82292e4705 HTTP/1.1" 200 None 2026-03-24 07:08:11,122 - lava_callback - DEBUG - Processing node 69bf954af1004e82292e4705: baseline-arm-clabbe 2026-03-24 07:08:11,122 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf954af1004e82292e4705 2026-03-24 07:08:11,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:08:11,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf954af1004e82292e4705 HTTP/1.1" 200 None 2026-03-24 07:08:11,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:08:11,154 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef76d5731cbd74145fb9 HTTP/1.1" 200 None 2026-03-24 07:08:11,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:08:11,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf954af1004e82292e4705 HTTP/1.1" 200 None 2026-03-24 07:08:11,183 - lava_callback - ERROR - Error processing callback for node 69bf954af1004e82292e4705: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54922 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58340 - "GET / HTTP/1.0" 200 OK 2026-03-24 07:08:51,200 - lava_callback - INFO - Received callback for node 69bf956df1004e82292e4715 INFO: 172.21.0.1:36048 - "POST /node/69bf956df1004e82292e4715 HTTP/1.0" 202 Accepted 2026-03-24 07:08:51,206 - lava_callback - INFO - Starting async job submit for node 69bf956df1004e82292e4715 2026-03-24 07:08:51,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:08:51,221 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf956df1004e82292e4715 HTTP/1.1" 200 None 2026-03-24 07:08:51,222 - lava_callback - DEBUG - Processing node 69bf956df1004e82292e4715: baseline-arm-clabbe 2026-03-24 07:08:51,222 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf956df1004e82292e4715 2026-03-24 07:08:51,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:08:51,236 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf956df1004e82292e4715 HTTP/1.1" 200 None 2026-03-24 07:08:51,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:08:51,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef76d5731cbd74145fb9 HTTP/1.1" 200 None 2026-03-24 07:08:51,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:08:51,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf956df1004e82292e4715 HTTP/1.1" 200 None 2026-03-24 07:08:51,347 - lava_callback - ERROR - Error processing callback for node 69bf956df1004e82292e4715: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33496 - "GET / HTTP/1.0" 200 OK 2026-03-24 07:09:11,160 - lava_callback - INFO - Received callback for node 69bf9591f1004e82292e4721 INFO: 172.21.0.1:41888 - "POST /node/69bf9591f1004e82292e4721 HTTP/1.0" 202 Accepted 2026-03-24 07:09:11,166 - lava_callback - INFO - Starting async job submit for node 69bf9591f1004e82292e4721 2026-03-24 07:09:11,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:09:11,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf9591f1004e82292e4721 HTTP/1.1" 200 None 2026-03-24 07:09:11,185 - lava_callback - DEBUG - Processing node 69bf9591f1004e82292e4721: baseline-arm-clabbe 2026-03-24 07:09:11,185 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf9591f1004e82292e4721 2026-03-24 07:09:11,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:09:11,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf9591f1004e82292e4721 HTTP/1.1" 200 None 2026-03-24 07:09:11,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:09:11,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef76d5731cbd74145fb9 HTTP/1.1" 200 None 2026-03-24 07:09:11,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:09:11,235 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf9591f1004e82292e4721 HTTP/1.1" 200 None 2026-03-24 07:09:11,237 - lava_callback - ERROR - Error processing callback for node 69bf9591f1004e82292e4721: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41900 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38428 - "GET / HTTP/1.0" 200 OK 2026-03-24 07:09:51,211 - lava_callback - INFO - Received callback for node 69bf95b5f1004e82292e4731 INFO: 172.21.0.1:59950 - "POST /node/69bf95b5f1004e82292e4731 HTTP/1.0" 202 Accepted 2026-03-24 07:09:51,216 - lava_callback - INFO - Starting async job submit for node 69bf95b5f1004e82292e4731 2026-03-24 07:09:51,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:09:51,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf95b5f1004e82292e4731 HTTP/1.1" 200 None 2026-03-24 07:09:51,242 - lava_callback - DEBUG - Processing node 69bf95b5f1004e82292e4731: baseline-arm-clabbe 2026-03-24 07:09:51,242 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf95b5f1004e82292e4731 2026-03-24 07:09:51,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:09:51,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf95b5f1004e82292e4731 HTTP/1.1" 200 None 2026-03-24 07:09:51,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:09:51,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef76d5731cbd74145fb9 HTTP/1.1" 200 None 2026-03-24 07:09:51,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:09:51,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf95b5f1004e82292e4731 HTTP/1.1" 200 None 2026-03-24 07:09:51,341 - lava_callback - ERROR - Error processing callback for node 69bf95b5f1004e82292e4731: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56554 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55594 - "GET / HTTP/1.0" 200 OK 2026-03-24 07:10:31,169 - lava_callback - INFO - Received callback for node 69bf95d9f1004e82292e4745 INFO: 172.21.0.1:35220 - "POST /node/69bf95d9f1004e82292e4745 HTTP/1.0" 202 Accepted 2026-03-24 07:10:31,175 - lava_callback - INFO - Starting async job submit for node 69bf95d9f1004e82292e4745 2026-03-24 07:10:31,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:10:31,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf95d9f1004e82292e4745 HTTP/1.1" 200 None 2026-03-24 07:10:31,193 - lava_callback - DEBUG - Processing node 69bf95d9f1004e82292e4745: baseline-arm-clabbe 2026-03-24 07:10:31,193 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf95d9f1004e82292e4745 2026-03-24 07:10:31,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:10:31,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf95d9f1004e82292e4745 HTTP/1.1" 200 None 2026-03-24 07:10:31,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:10:31,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef76d5731cbd74145fb9 HTTP/1.1" 200 None 2026-03-24 07:10:31,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:10:31,254 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf95d9f1004e82292e4745 HTTP/1.1" 200 None 2026-03-24 07:10:31,256 - lava_callback - ERROR - Error processing callback for node 69bf95d9f1004e82292e4745: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51544 - "GET / HTTP/1.0" 200 OK 2026-03-24 07:11:11,226 - lava_callback - INFO - Received callback for node 69bf95fdf1004e82292e4756 INFO: 172.21.0.1:36054 - "POST /node/69bf95fdf1004e82292e4756 HTTP/1.0" 202 Accepted 2026-03-24 07:11:11,232 - lava_callback - INFO - Starting async job submit for node 69bf95fdf1004e82292e4756 2026-03-24 07:11:11,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:11:11,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf95fdf1004e82292e4756 HTTP/1.1" 200 None 2026-03-24 07:11:11,252 - lava_callback - DEBUG - Processing node 69bf95fdf1004e82292e4756: baseline-arm-clabbe 2026-03-24 07:11:11,252 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf95fdf1004e82292e4756 2026-03-24 07:11:11,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:11:11,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf95fdf1004e82292e4756 HTTP/1.1" 200 None 2026-03-24 07:11:11,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:11:11,286 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef76d5731cbd74145fb9 HTTP/1.1" 200 None 2026-03-24 07:11:11,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:11:11,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf95fdf1004e82292e4756 HTTP/1.1" 200 None 2026-03-24 07:11:11,309 - lava_callback - ERROR - Error processing callback for node 69bf95fdf1004e82292e4756: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36062 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42640 - "GET / HTTP/1.0" 200 OK 2026-03-24 07:11:51,242 - lava_callback - INFO - Received callback for node 69bf9621f1004e82292e4774 INFO: 172.21.0.1:53342 - "POST /node/69bf9621f1004e82292e4774 HTTP/1.0" 202 Accepted 2026-03-24 07:11:51,248 - lava_callback - INFO - Starting async job submit for node 69bf9621f1004e82292e4774 2026-03-24 07:11:51,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:11:51,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf9621f1004e82292e4774 HTTP/1.1" 200 None 2026-03-24 07:11:51,332 - lava_callback - DEBUG - Processing node 69bf9621f1004e82292e4774: baseline-arm-clabbe 2026-03-24 07:11:51,332 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf9621f1004e82292e4774 2026-03-24 07:11:51,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:11:51,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf9621f1004e82292e4774 HTTP/1.1" 200 None 2026-03-24 07:11:51,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:11:51,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef76d5731cbd74145fb9 HTTP/1.1" 200 None 2026-03-24 07:11:51,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:11:51,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf9621f1004e82292e4774 HTTP/1.1" 200 None 2026-03-24 07:11:51,405 - lava_callback - ERROR - Error processing callback for node 69bf9621f1004e82292e4774: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55822 - "GET / HTTP/1.0" 200 OK 2026-03-24 07:12:11,313 - lava_callback - INFO - Received callback for node 69bf9645f1004e82292e4789 INFO: 172.21.0.1:43708 - "POST /node/69bf9645f1004e82292e4789 HTTP/1.0" 202 Accepted 2026-03-24 07:12:11,318 - lava_callback - INFO - Starting async job submit for node 69bf9645f1004e82292e4789 2026-03-24 07:12:11,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:12:11,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf9645f1004e82292e4789 HTTP/1.1" 200 None 2026-03-24 07:12:11,500 - lava_callback - DEBUG - Processing node 69bf9645f1004e82292e4789: baseline-arm-clabbe 2026-03-24 07:12:11,500 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf9645f1004e82292e4789 2026-03-24 07:12:11,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:12:11,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf9645f1004e82292e4789 HTTP/1.1" 200 None 2026-03-24 07:12:11,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:12:11,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef76d5731cbd74145fb9 HTTP/1.1" 200 None 2026-03-24 07:12:11,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:12:11,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf9645f1004e82292e4789 HTTP/1.1" 200 None 2026-03-24 07:12:11,564 - lava_callback - ERROR - Error processing callback for node 69bf9645f1004e82292e4789: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43730 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44340 - "GET / HTTP/1.0" 200 OK 2026-03-24 07:12:51,297 - lava_callback - INFO - Received callback for node 69bf9669f1004e82292e479a INFO: 172.21.0.1:34170 - "POST /node/69bf9669f1004e82292e479a HTTP/1.0" 202 Accepted 2026-03-24 07:12:51,302 - lava_callback - INFO - Starting async job submit for node 69bf9669f1004e82292e479a 2026-03-24 07:12:51,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:12:51,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf9669f1004e82292e479a HTTP/1.1" 200 None 2026-03-24 07:12:51,319 - lava_callback - DEBUG - Processing node 69bf9669f1004e82292e479a: baseline-arm-clabbe 2026-03-24 07:12:51,319 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf9669f1004e82292e479a 2026-03-24 07:12:51,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:12:51,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf9669f1004e82292e479a HTTP/1.1" 200 None 2026-03-24 07:12:51,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:12:51,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef76d5731cbd74145fb9 HTTP/1.1" 200 None 2026-03-24 07:12:51,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:12:51,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf9669f1004e82292e479a HTTP/1.1" 200 None 2026-03-24 07:12:51,400 - lava_callback - ERROR - Error processing callback for node 69bf9669f1004e82292e479a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44564 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47132 - "GET / HTTP/1.0" 200 OK 2026-03-24 07:13:31,325 - lava_callback - INFO - Received callback for node 69bf968df1004e82292e47a1 INFO: 172.21.0.1:35312 - "POST /node/69bf968df1004e82292e47a1 HTTP/1.0" 202 Accepted 2026-03-24 07:13:31,330 - lava_callback - INFO - Starting async job submit for node 69bf968df1004e82292e47a1 2026-03-24 07:13:31,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:13:31,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf968df1004e82292e47a1 HTTP/1.1" 200 None 2026-03-24 07:13:31,346 - lava_callback - DEBUG - Processing node 69bf968df1004e82292e47a1: baseline-arm-clabbe 2026-03-24 07:13:31,346 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf968df1004e82292e47a1 2026-03-24 07:13:31,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:13:31,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf968df1004e82292e47a1 HTTP/1.1" 200 None 2026-03-24 07:13:31,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:13:31,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef76d5731cbd74145fb9 HTTP/1.1" 200 None 2026-03-24 07:13:31,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:13:31,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf968df1004e82292e47a1 HTTP/1.1" 200 None 2026-03-24 07:13:31,395 - lava_callback - ERROR - Error processing callback for node 69bf968df1004e82292e47a1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36472 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57906 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36486 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33714 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51508 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60640 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47568 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55870 - "GET / HTTP/1.0" 200 OK 2026-03-24 07:20:34,530 - lava_callback - INFO - Received callback for node 69bf982df1004e82292e47e7 INFO: 172.21.0.1:41866 - "POST /node/69bf982df1004e82292e47e7 HTTP/1.0" 202 Accepted 2026-03-24 07:20:34,536 - lava_callback - INFO - Starting async job submit for node 69bf982df1004e82292e47e7 2026-03-24 07:20:34,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:20:34,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf982df1004e82292e47e7 HTTP/1.1" 200 None 2026-03-24 07:20:34,568 - lava_callback - DEBUG - Processing node 69bf982df1004e82292e47e7: baseline-arm64-baylibre 2026-03-24 07:20:34,568 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf982df1004e82292e47e7 2026-03-24 07:20:34,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:20:34,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf982df1004e82292e47e7 HTTP/1.1" 200 None 2026-03-24 07:20:34,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:20:34,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-24 07:20:34,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:20:34,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf982df1004e82292e47e7 HTTP/1.1" 200 None 2026-03-24 07:20:34,657 - lava_callback - ERROR - Error processing callback for node 69bf982df1004e82292e47e7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37998 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45432 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41522 - "GET / HTTP/1.0" 200 OK 2026-03-24 07:22:54,571 - lava_callback - INFO - Received callback for node 69bf98c6f1004e82292e47fb INFO: 172.21.0.1:53040 - "POST /node/69bf98c6f1004e82292e47fb HTTP/1.0" 202 Accepted 2026-03-24 07:22:54,576 - lava_callback - INFO - Starting async job submit for node 69bf98c6f1004e82292e47fb 2026-03-24 07:22:54,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:22:54,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf98c6f1004e82292e47fb HTTP/1.1" 200 None 2026-03-24 07:22:54,592 - lava_callback - DEBUG - Processing node 69bf98c6f1004e82292e47fb: baseline-arm64-baylibre 2026-03-24 07:22:54,592 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf98c6f1004e82292e47fb 2026-03-24 07:22:54,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:22:54,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf98c6f1004e82292e47fb HTTP/1.1" 200 None 2026-03-24 07:22:54,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:22:54,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-24 07:22:54,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:22:54,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf98c6f1004e82292e47fb HTTP/1.1" 200 None 2026-03-24 07:22:54,665 - lava_callback - ERROR - Error processing callback for node 69bf98c6f1004e82292e47fb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53608 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35394 - "GET / HTTP/1.0" 200 OK 2026-03-24 07:24:11,849 - lava_callback - INFO - Received callback for node 69bf9917f1004e82292e4805 INFO: 172.21.0.1:51880 - "POST /node/69bf9917f1004e82292e4805 HTTP/1.0" 202 Accepted 2026-03-24 07:24:11,854 - lava_callback - INFO - Starting async job submit for node 69bf9917f1004e82292e4805 2026-03-24 07:24:11,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:24:11,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf9917f1004e82292e4805 HTTP/1.1" 200 None 2026-03-24 07:24:11,872 - lava_callback - DEBUG - Processing node 69bf9917f1004e82292e4805: baseline-arm64-clabbe 2026-03-24 07:24:11,872 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf9917f1004e82292e4805 2026-03-24 07:24:11,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:24:11,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf9917f1004e82292e4805 HTTP/1.1" 200 None 2026-03-24 07:24:11,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:24:11,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-24 07:24:11,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:24:11,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf9917f1004e82292e4805 HTTP/1.1" 200 None 2026-03-24 07:24:11,938 - lava_callback - ERROR - Error processing callback for node 69bf9917f1004e82292e4805: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51904 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49444 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60144 - "GET / HTTP/1.0" 200 OK 2026-03-24 07:25:31,829 - lava_callback - INFO - Received callback for node 69bf9963f1004e82292e4813 INFO: 172.21.0.1:46778 - "POST /node/69bf9963f1004e82292e4813 HTTP/1.0" 202 Accepted 2026-03-24 07:25:31,835 - lava_callback - INFO - Starting async job submit for node 69bf9963f1004e82292e4813 2026-03-24 07:25:31,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:25:31,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf9963f1004e82292e4813 HTTP/1.1" 200 None 2026-03-24 07:25:31,852 - lava_callback - DEBUG - Processing node 69bf9963f1004e82292e4813: baseline-arm64-clabbe 2026-03-24 07:25:31,852 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf9963f1004e82292e4813 2026-03-24 07:25:31,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:25:31,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf9963f1004e82292e4813 HTTP/1.1" 200 None 2026-03-24 07:25:31,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:25:31,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-24 07:25:31,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:25:31,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf9963f1004e82292e4813 HTTP/1.1" 200 None 2026-03-24 07:25:31,917 - lava_callback - ERROR - Error processing callback for node 69bf9963f1004e82292e4813: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46452 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35494 - "GET / HTTP/1.0" 200 OK 2026-03-24 07:26:51,891 - lava_callback - INFO - Received callback for node 69bf99aef1004e82292e4824 INFO: 172.21.0.1:60626 - "POST /node/69bf99aef1004e82292e4824 HTTP/1.0" 202 Accepted 2026-03-24 07:26:51,897 - lava_callback - INFO - Starting async job submit for node 69bf99aef1004e82292e4824 2026-03-24 07:26:51,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:26:51,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf99aef1004e82292e4824 HTTP/1.1" 200 None 2026-03-24 07:26:51,914 - lava_callback - DEBUG - Processing node 69bf99aef1004e82292e4824: baseline-arm64-clabbe 2026-03-24 07:26:51,914 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf99aef1004e82292e4824 2026-03-24 07:26:51,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:26:51,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf99aef1004e82292e4824 HTTP/1.1" 200 None 2026-03-24 07:26:51,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:26:51,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-24 07:26:51,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:26:51,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf99aef1004e82292e4824 HTTP/1.1" 200 None 2026-03-24 07:26:51,966 - lava_callback - ERROR - Error processing callback for node 69bf99aef1004e82292e4824: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49886 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54626 - "GET / HTTP/1.0" 200 OK 2026-03-24 07:28:11,931 - lava_callback - INFO - Received callback for node 69bf99f9f1004e82292e482d INFO: 172.21.0.1:54912 - "POST /node/69bf99f9f1004e82292e482d HTTP/1.0" 202 Accepted 2026-03-24 07:28:11,939 - lava_callback - INFO - Starting async job submit for node 69bf99f9f1004e82292e482d 2026-03-24 07:28:11,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:28:11,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf99f9f1004e82292e482d HTTP/1.1" 200 None 2026-03-24 07:28:11,963 - lava_callback - DEBUG - Processing node 69bf99f9f1004e82292e482d: baseline-arm64-clabbe 2026-03-24 07:28:11,963 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf99f9f1004e82292e482d 2026-03-24 07:28:11,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:28:11,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf99f9f1004e82292e482d HTTP/1.1" 200 None 2026-03-24 07:28:11,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:28:11,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-24 07:28:12,000 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:28:12,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf99f9f1004e82292e482d HTTP/1.1" 200 None 2026-03-24 07:28:12,032 - lava_callback - ERROR - Error processing callback for node 69bf99f9f1004e82292e482d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54928 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42908 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.21.0.1:42912 - "GET / HTTP/1.0" 200 OK 2026-03-24 07:29:12,019 - lava_callback - INFO - Received callback for node 69bf9a44f1004e82292e4841 INFO: 172.21.0.1:35122 - "POST /node/69bf9a44f1004e82292e4841 HTTP/1.0" 202 Accepted 2026-03-24 07:29:12,025 - lava_callback - INFO - Starting async job submit for node 69bf9a44f1004e82292e4841 2026-03-24 07:29:12,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:29:12,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf9a44f1004e82292e4841 HTTP/1.1" 200 None 2026-03-24 07:29:12,041 - lava_callback - DEBUG - Processing node 69bf9a44f1004e82292e4841: baseline-arm64-clabbe 2026-03-24 07:29:12,041 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf9a44f1004e82292e4841 2026-03-24 07:29:12,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:29:12,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf9a44f1004e82292e4841 HTTP/1.1" 200 None 2026-03-24 07:29:12,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:29:12,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-24 07:29:12,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:29:12,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf9a44f1004e82292e4841 HTTP/1.1" 200 None 2026-03-24 07:29:12,102 - lava_callback - ERROR - Error processing callback for node 69bf9a44f1004e82292e4841: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35136 - "GET /login HTTP/1.0" 404 Not Found INFO: 172.21.0.1:35146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35160 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38884 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56974 - "GET / HTTP/1.0" 200 OK 2026-03-24 07:30:31,999 - lava_callback - INFO - Received callback for node 69bf9a8ff1004e82292e4848 INFO: 172.21.0.1:44020 - "POST /node/69bf9a8ff1004e82292e4848 HTTP/1.0" 202 Accepted 2026-03-24 07:30:32,005 - lava_callback - INFO - Starting async job submit for node 69bf9a8ff1004e82292e4848 2026-03-24 07:30:32,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:30:32,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf9a8ff1004e82292e4848 HTTP/1.1" 200 None 2026-03-24 07:30:32,021 - lava_callback - DEBUG - Processing node 69bf9a8ff1004e82292e4848: baseline-arm64-clabbe 2026-03-24 07:30:32,021 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf9a8ff1004e82292e4848 2026-03-24 07:30:32,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:30:32,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf9a8ff1004e82292e4848 HTTP/1.1" 200 None 2026-03-24 07:30:32,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:30:32,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-24 07:30:32,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:30:32,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf9a8ff1004e82292e4848 HTTP/1.1" 200 None 2026-03-24 07:30:32,091 - lava_callback - ERROR - Error processing callback for node 69bf9a8ff1004e82292e4848: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36352 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52324 - "GET / HTTP/1.0" 200 OK 2026-03-24 07:31:35,217 - lava_callback - INFO - Received callback for node 69bf9ad5f1004e82292e4851 INFO: 172.21.0.1:58112 - "POST /node/69bf9ad5f1004e82292e4851 HTTP/1.0" 202 Accepted 2026-03-24 07:31:35,222 - lava_callback - INFO - Starting async job submit for node 69bf9ad5f1004e82292e4851 2026-03-24 07:31:35,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:31:35,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf9ad5f1004e82292e4851 HTTP/1.1" 200 None 2026-03-24 07:31:35,238 - lava_callback - DEBUG - Processing node 69bf9ad5f1004e82292e4851: baseline-arm64-baylibre 2026-03-24 07:31:35,238 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf9ad5f1004e82292e4851 2026-03-24 07:31:35,239 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:31:35,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf9ad5f1004e82292e4851 HTTP/1.1" 200 None 2026-03-24 07:31:35,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:31:35,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21b717e5cb936602bc2b HTTP/1.1" 200 None 2026-03-24 07:31:35,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:31:35,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf9ad5f1004e82292e4851 HTTP/1.1" 200 None 2026-03-24 07:31:35,305 - lava_callback - ERROR - Error processing callback for node 69bf9ad5f1004e82292e4851: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54012 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37252 - "GET / HTTP/1.0" 200 OK 2026-03-24 07:32:55,213 - lava_callback - INFO - Received callback for node 69bf9b1cf1004e82292e4863 INFO: 172.21.0.1:40302 - "POST /node/69bf9b1cf1004e82292e4863 HTTP/1.0" 202 Accepted 2026-03-24 07:32:55,218 - lava_callback - INFO - Starting async job submit for node 69bf9b1cf1004e82292e4863 2026-03-24 07:32:55,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:32:55,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf9b1cf1004e82292e4863 HTTP/1.1" 200 None 2026-03-24 07:32:55,235 - lava_callback - DEBUG - Processing node 69bf9b1cf1004e82292e4863: baseline-arm64-baylibre 2026-03-24 07:32:55,235 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf9b1cf1004e82292e4863 2026-03-24 07:32:55,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:32:55,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf9b1cf1004e82292e4863 HTTP/1.1" 200 None 2026-03-24 07:32:55,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:32:55,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21b717e5cb936602bc2b HTTP/1.1" 200 None 2026-03-24 07:32:55,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:32:55,288 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf9b1cf1004e82292e4863 HTTP/1.1" 200 None 2026-03-24 07:32:55,295 - lava_callback - ERROR - Error processing callback for node 69bf9b1cf1004e82292e4863: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59132 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45808 - "GET / HTTP/1.0" 200 OK 2026-03-24 07:34:15,104 - lava_callback - INFO - Received callback for node 69bf9b69f1004e82292e486d INFO: 172.21.0.1:45822 - "POST /node/69bf9b69f1004e82292e486d HTTP/1.0" 202 Accepted 2026-03-24 07:34:15,109 - lava_callback - INFO - Starting async job submit for node 69bf9b69f1004e82292e486d 2026-03-24 07:34:15,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:34:15,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf9b69f1004e82292e486d HTTP/1.1" 200 None 2026-03-24 07:34:15,148 - lava_callback - DEBUG - Processing node 69bf9b69f1004e82292e486d: baseline-arm64-baylibre 2026-03-24 07:34:15,148 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf9b69f1004e82292e486d 2026-03-24 07:34:15,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:34:15,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf9b69f1004e82292e486d HTTP/1.1" 200 None 2026-03-24 07:34:15,163 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:45836 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-24 07:34:15,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21b717e5cb936602bc2b HTTP/1.1" 200 None 2026-03-24 07:34:15,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:34:15,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf9b69f1004e82292e486d HTTP/1.1" 200 None 2026-03-24 07:34:15,199 - lava_callback - ERROR - Error processing callback for node 69bf9b69f1004e82292e486d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36564 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60972 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51592 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52390 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47232 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57402 - "GET / HTTP/1.0" 200 OK 2026-03-24 07:39:52,412 - lava_callback - INFO - Received callback for node 69bf9cb7f1004e82292e4889 INFO: 172.21.0.1:47726 - "POST /node/69bf9cb7f1004e82292e4889 HTTP/1.0" 202 Accepted 2026-03-24 07:39:52,418 - lava_callback - INFO - Starting async job submit for node 69bf9cb7f1004e82292e4889 2026-03-24 07:39:52,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:39:52,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf9cb7f1004e82292e4889 HTTP/1.1" 200 None 2026-03-24 07:39:52,434 - lava_callback - DEBUG - Processing node 69bf9cb7f1004e82292e4889: baseline-arm64-clabbe 2026-03-24 07:39:52,434 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf9cb7f1004e82292e4889 2026-03-24 07:39:52,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:39:52,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf9cb7f1004e82292e4889 HTTP/1.1" 200 None 2026-03-24 07:39:52,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:39:52,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21b717e5cb936602bc2b HTTP/1.1" 200 None 2026-03-24 07:39:52,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:39:52,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf9cb7f1004e82292e4889 HTTP/1.1" 200 None 2026-03-24 07:39:52,531 - lava_callback - ERROR - Error processing callback for node 69bf9cb7f1004e82292e4889: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41432 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51818 - "GET / HTTP/1.0" 200 OK 2026-03-24 07:40:55,629 - lava_callback - INFO - Received callback for node 69bf9cfff1004e82292e489a INFO: 172.21.0.1:60762 - "POST /node/69bf9cfff1004e82292e489a HTTP/1.0" 202 Accepted 2026-03-24 07:40:55,634 - lava_callback - INFO - Starting async job submit for node 69bf9cfff1004e82292e489a 2026-03-24 07:40:55,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:40:55,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf9cfff1004e82292e489a HTTP/1.1" 200 None 2026-03-24 07:40:55,652 - lava_callback - DEBUG - Processing node 69bf9cfff1004e82292e489a: baseline-arm64-baylibre 2026-03-24 07:40:55,652 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf9cfff1004e82292e489a 2026-03-24 07:40:55,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:40:55,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf9cfff1004e82292e489a HTTP/1.1" 200 None 2026-03-24 07:40:55,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:40:55,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-24 07:40:55,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:40:55,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf9cfff1004e82292e489a HTTP/1.1" 200 None 2026-03-24 07:40:55,732 - lava_callback - ERROR - Error processing callback for node 69bf9cfff1004e82292e489a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58882 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39086 - "GET / HTTP/1.0" 200 OK 2026-03-24 07:42:12,541 - lava_callback - INFO - Received callback for node 69bf9d4ff1004e82292e48ad INFO: 172.21.0.1:58642 - "POST /node/69bf9d4ff1004e82292e48ad HTTP/1.0" 202 Accepted 2026-03-24 07:42:12,546 - lava_callback - INFO - Starting async job submit for node 69bf9d4ff1004e82292e48ad 2026-03-24 07:42:12,548 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:42:12,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf9d4ff1004e82292e48ad HTTP/1.1" 200 None 2026-03-24 07:42:12,569 - lava_callback - DEBUG - Processing node 69bf9d4ff1004e82292e48ad: baseline-arm64-clabbe 2026-03-24 07:42:12,570 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf9d4ff1004e82292e48ad 2026-03-24 07:42:12,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:42:12,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf9d4ff1004e82292e48ad HTTP/1.1" 200 None 2026-03-24 07:42:12,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:42:12,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-24 07:42:12,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:42:12,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf9d4ff1004e82292e48ad HTTP/1.1" 200 None 2026-03-24 07:42:12,629 - lava_callback - ERROR - Error processing callback for node 69bf9d4ff1004e82292e48ad: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58652 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38712 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43634 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43322 - "GET / HTTP/1.0" 200 OK 2026-03-24 07:44:52,658 - lava_callback - INFO - Received callback for node 69bf9de4f1004e82292e48d8 INFO: 172.21.0.1:37844 - "POST /node/69bf9de4f1004e82292e48d8 HTTP/1.0" 202 Accepted 2026-03-24 07:44:52,663 - lava_callback - INFO - Starting async job submit for node 69bf9de4f1004e82292e48d8 2026-03-24 07:44:52,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:44:52,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf9de4f1004e82292e48d8 HTTP/1.1" 200 None 2026-03-24 07:44:52,690 - lava_callback - DEBUG - Processing node 69bf9de4f1004e82292e48d8: baseline-arm64-clabbe 2026-03-24 07:44:52,690 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf9de4f1004e82292e48d8 2026-03-24 07:44:52,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:44:52,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf9de4f1004e82292e48d8 HTTP/1.1" 200 None 2026-03-24 07:44:52,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:44:52,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-24 07:44:52,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:44:52,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf9de4f1004e82292e48d8 HTTP/1.1" 200 None 2026-03-24 07:44:52,768 - lava_callback - ERROR - Error processing callback for node 69bf9de4f1004e82292e48d8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33886 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59048 - "GET / HTTP/1.0" 200 OK 2026-03-24 07:46:12,739 - lava_callback - INFO - Received callback for node 69bf9e30f1004e82292e48ec INFO: 172.21.0.1:59586 - "POST /node/69bf9e30f1004e82292e48ec HTTP/1.0" 202 Accepted 2026-03-24 07:46:12,744 - lava_callback - INFO - Starting async job submit for node 69bf9e30f1004e82292e48ec 2026-03-24 07:46:12,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:46:12,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf9e30f1004e82292e48ec HTTP/1.1" 200 None 2026-03-24 07:46:12,760 - lava_callback - DEBUG - Processing node 69bf9e30f1004e82292e48ec: baseline-arm64-clabbe 2026-03-24 07:46:12,761 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf9e30f1004e82292e48ec 2026-03-24 07:46:12,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:46:12,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf9e30f1004e82292e48ec HTTP/1.1" 200 None 2026-03-24 07:46:12,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:46:12,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21b717e5cb936602bc2b HTTP/1.1" 200 None 2026-03-24 07:46:12,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:46:12,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf9e30f1004e82292e48ec HTTP/1.1" 200 None 2026-03-24 07:46:12,821 - lava_callback - ERROR - Error processing callback for node 69bf9e30f1004e82292e48ec: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59590 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43264 - "GET / HTTP/1.0" 200 OK 2026-03-24 07:47:12,805 - lava_callback - INFO - Received callback for node 69bf9e7af1004e82292e48f9 INFO: 172.21.0.1:55186 - "POST /node/69bf9e7af1004e82292e48f9 HTTP/1.0" 202 Accepted 2026-03-24 07:47:12,810 - lava_callback - INFO - Starting async job submit for node 69bf9e7af1004e82292e48f9 2026-03-24 07:47:12,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:47:12,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf9e7af1004e82292e48f9 HTTP/1.1" 200 None 2026-03-24 07:47:12,826 - lava_callback - DEBUG - Processing node 69bf9e7af1004e82292e48f9: baseline-arm64-clabbe 2026-03-24 07:47:12,826 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf9e7af1004e82292e48f9 2026-03-24 07:47:12,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:47:12,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf9e7af1004e82292e48f9 HTTP/1.1" 200 None 2026-03-24 07:47:12,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:47:12,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-24 07:47:12,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:47:12,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf9e7af1004e82292e48f9 HTTP/1.1" 200 None 2026-03-24 07:47:12,884 - lava_callback - ERROR - Error processing callback for node 69bf9e7af1004e82292e48f9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55200 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40190 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35694 - "GET / HTTP/1.0" 200 OK 2026-03-24 07:48:32,837 - lava_callback - INFO - Received callback for node 69bf9ec7f1004e82292e490d INFO: 172.21.0.1:51656 - "POST /node/69bf9ec7f1004e82292e490d HTTP/1.0" 202 Accepted 2026-03-24 07:48:32,842 - lava_callback - INFO - Starting async job submit for node 69bf9ec7f1004e82292e490d 2026-03-24 07:48:32,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:48:32,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf9ec7f1004e82292e490d HTTP/1.1" 200 None 2026-03-24 07:48:32,858 - lava_callback - DEBUG - Processing node 69bf9ec7f1004e82292e490d: baseline-arm64-clabbe 2026-03-24 07:48:32,859 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf9ec7f1004e82292e490d 2026-03-24 07:48:32,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:48:32,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf9ec7f1004e82292e490d HTTP/1.1" 200 None 2026-03-24 07:48:32,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:48:32,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21b717e5cb936602bc2b HTTP/1.1" 200 None 2026-03-24 07:48:32,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:48:32,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf9ec7f1004e82292e490d HTTP/1.1" 200 None 2026-03-24 07:48:32,925 - lava_callback - ERROR - Error processing callback for node 69bf9ec7f1004e82292e490d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46286 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47402 - "GET / HTTP/1.0" 200 OK 2026-03-24 07:49:52,889 - lava_callback - INFO - Received callback for node 69bf9f14f1004e82292e4925 INFO: 172.21.0.1:33864 - "POST /node/69bf9f14f1004e82292e4925 HTTP/1.0" 202 Accepted 2026-03-24 07:49:52,894 - lava_callback - INFO - Starting async job submit for node 69bf9f14f1004e82292e4925 2026-03-24 07:49:52,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:49:52,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf9f14f1004e82292e4925 HTTP/1.1" 200 None 2026-03-24 07:49:52,911 - lava_callback - DEBUG - Processing node 69bf9f14f1004e82292e4925: baseline-arm64-clabbe 2026-03-24 07:49:52,911 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf9f14f1004e82292e4925 2026-03-24 07:49:52,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:49:52,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf9f14f1004e82292e4925 HTTP/1.1" 200 None 2026-03-24 07:49:52,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:49:52,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-24 07:49:52,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:49:53,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf9f14f1004e82292e4925 HTTP/1.1" 200 None 2026-03-24 07:49:53,018 - lava_callback - ERROR - Error processing callback for node 69bf9f14f1004e82292e4925: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37276 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44922 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40940 - "GET / HTTP/1.0" 200 OK 2026-03-24 07:52:13,012 - lava_callback - INFO - Received callback for node 69bf9fa8f1004e82292e4937 INFO: 172.21.0.1:50736 - "POST /node/69bf9fa8f1004e82292e4937 HTTP/1.0" 202 Accepted 2026-03-24 07:52:13,018 - lava_callback - INFO - Starting async job submit for node 69bf9fa8f1004e82292e4937 2026-03-24 07:52:13,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:52:13,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf9fa8f1004e82292e4937 HTTP/1.1" 200 None 2026-03-24 07:52:13,039 - lava_callback - DEBUG - Processing node 69bf9fa8f1004e82292e4937: baseline-arm64-clabbe 2026-03-24 07:52:13,039 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf9fa8f1004e82292e4937 2026-03-24 07:52:13,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:52:13,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf9fa8f1004e82292e4937 HTTP/1.1" 200 None 2026-03-24 07:52:13,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:52:13,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-24 07:52:13,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:52:13,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf9fa8f1004e82292e4937 HTTP/1.1" 200 None 2026-03-24 07:52:13,097 - lava_callback - ERROR - Error processing callback for node 69bf9fa8f1004e82292e4937: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50740 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53896 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47714 - "GET / HTTP/1.0" 200 OK 2026-03-24 07:53:33,011 - lava_callback - INFO - Received callback for node 69bf9ff3f1004e82292e494a INFO: 172.21.0.1:56656 - "POST /node/69bf9ff3f1004e82292e494a HTTP/1.0" 202 Accepted 2026-03-24 07:53:33,016 - lava_callback - INFO - Starting async job submit for node 69bf9ff3f1004e82292e494a 2026-03-24 07:53:33,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:53:33,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf9ff3f1004e82292e494a HTTP/1.1" 200 None 2026-03-24 07:53:33,032 - lava_callback - DEBUG - Processing node 69bf9ff3f1004e82292e494a: baseline-arm64-clabbe 2026-03-24 07:53:33,033 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bf9ff3f1004e82292e494a 2026-03-24 07:53:33,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:53:33,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf9ff3f1004e82292e494a HTTP/1.1" 200 None 2026-03-24 07:53:33,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:53:33,105 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21b717e5cb936602bc2b HTTP/1.1" 200 None 2026-03-24 07:53:33,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:53:33,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bf9ff3f1004e82292e494a HTTP/1.1" 200 None 2026-03-24 07:53:33,135 - lava_callback - ERROR - Error processing callback for node 69bf9ff3f1004e82292e494a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60644 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40856 - "GET / HTTP/1.0" 200 OK 2026-03-24 07:54:53,118 - lava_callback - INFO - Received callback for node 69bfa03ef1004e82292e495b INFO: 172.21.0.1:39434 - "POST /node/69bfa03ef1004e82292e495b HTTP/1.0" 202 Accepted 2026-03-24 07:54:53,124 - lava_callback - INFO - Starting async job submit for node 69bfa03ef1004e82292e495b 2026-03-24 07:54:53,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:54:53,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa03ef1004e82292e495b HTTP/1.1" 200 None 2026-03-24 07:54:53,143 - lava_callback - DEBUG - Processing node 69bfa03ef1004e82292e495b: baseline-arm64-clabbe 2026-03-24 07:54:53,143 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfa03ef1004e82292e495b 2026-03-24 07:54:53,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:54:53,158 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa03ef1004e82292e495b HTTP/1.1" 200 None 2026-03-24 07:54:53,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:54:53,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-24 07:54:53,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:54:53,195 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfa03ef1004e82292e495b HTTP/1.1" 200 None 2026-03-24 07:54:53,201 - lava_callback - ERROR - Error processing callback for node 69bfa03ef1004e82292e495b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32882 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:32890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51622 - "GET / HTTP/1.0" 200 OK 2026-03-24 07:56:13,192 - lava_callback - INFO - Received callback for node 69bfa088f1004e82292e496f INFO: 172.21.0.1:56512 - "POST /node/69bfa088f1004e82292e496f HTTP/1.0" 202 Accepted 2026-03-24 07:56:13,198 - lava_callback - INFO - Starting async job submit for node 69bfa088f1004e82292e496f 2026-03-24 07:56:13,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:56:13,221 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa088f1004e82292e496f HTTP/1.1" 200 None 2026-03-24 07:56:13,222 - lava_callback - DEBUG - Processing node 69bfa088f1004e82292e496f: baseline-arm64-clabbe 2026-03-24 07:56:13,223 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfa088f1004e82292e496f 2026-03-24 07:56:13,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:56:13,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa088f1004e82292e496f HTTP/1.1" 200 None 2026-03-24 07:56:13,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:56:13,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21b717e5cb936602bc2b HTTP/1.1" 200 None 2026-03-24 07:56:13,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:56:13,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfa088f1004e82292e496f HTTP/1.1" 200 None 2026-03-24 07:56:13,282 - lava_callback - ERROR - Error processing callback for node 69bfa088f1004e82292e496f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56528 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33324 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.21.0.1:33336 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33360 - "GET / HTTP/1.0" 200 OK 2026-03-24 07:57:16,631 - lava_callback - INFO - Received callback for node 69bfa0cdf1004e82292e497a INFO: 172.21.0.1:33372 - "POST /node/69bfa0cdf1004e82292e497a HTTP/1.0" 202 Accepted 2026-03-24 07:57:16,636 - lava_callback - INFO - Starting async job submit for node 69bfa0cdf1004e82292e497a 2026-03-24 07:57:16,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:57:16,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa0cdf1004e82292e497a HTTP/1.1" 200 None 2026-03-24 07:57:16,652 - lava_callback - DEBUG - Processing node 69bfa0cdf1004e82292e497a: baseline-arm64-baylibre 2026-03-24 07:57:16,652 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfa0cdf1004e82292e497a 2026-03-24 07:57:16,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:57:16,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa0cdf1004e82292e497a HTTP/1.1" 200 None 2026-03-24 07:57:16,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:57:16,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21b717e5cb936602bc2b HTTP/1.1" 200 None 2026-03-24 07:57:16,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:57:16,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfa0cdf1004e82292e497a HTTP/1.1" 200 None 2026-03-24 07:57:16,704 - lava_callback - ERROR - Error processing callback for node 69bfa0cdf1004e82292e497a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48806 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34908 - "GET / HTTP/1.0" 200 OK 2026-03-24 07:58:33,238 - lava_callback - INFO - Received callback for node 69bfa11df1004e82292e498b INFO: 172.21.0.1:45270 - "POST /node/69bfa11df1004e82292e498b HTTP/1.0" 202 Accepted 2026-03-24 07:58:33,244 - lava_callback - INFO - Starting async job submit for node 69bfa11df1004e82292e498b 2026-03-24 07:58:33,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:58:33,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa11df1004e82292e498b HTTP/1.1" 200 None 2026-03-24 07:58:33,263 - lava_callback - DEBUG - Processing node 69bfa11df1004e82292e498b: baseline-arm64-clabbe 2026-03-24 07:58:33,263 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfa11df1004e82292e498b 2026-03-24 07:58:33,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:58:33,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa11df1004e82292e498b HTTP/1.1" 200 None 2026-03-24 07:58:33,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:58:33,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21b717e5cb936602bc2b HTTP/1.1" 200 None 2026-03-24 07:58:33,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:58:33,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfa11df1004e82292e498b HTTP/1.1" 200 None 2026-03-24 07:58:33,328 - lava_callback - ERROR - Error processing callback for node 69bfa11df1004e82292e498b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52024 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43266 - "GET / HTTP/1.0" 200 OK 2026-03-24 07:59:53,311 - lava_callback - INFO - Received callback for node 69bfa168f1004e82292e499a INFO: 172.21.0.1:39860 - "POST /node/69bfa168f1004e82292e499a HTTP/1.0" 202 Accepted 2026-03-24 07:59:53,317 - lava_callback - INFO - Starting async job submit for node 69bfa168f1004e82292e499a 2026-03-24 07:59:53,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:59:53,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa168f1004e82292e499a HTTP/1.1" 200 None 2026-03-24 07:59:53,332 - lava_callback - DEBUG - Processing node 69bfa168f1004e82292e499a: baseline-arm64-clabbe 2026-03-24 07:59:53,333 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfa168f1004e82292e499a 2026-03-24 07:59:53,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:59:53,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa168f1004e82292e499a HTTP/1.1" 200 None 2026-03-24 07:59:53,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:59:53,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21b717e5cb936602bc2b HTTP/1.1" 200 None 2026-03-24 07:59:53,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:59:53,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfa168f1004e82292e499a HTTP/1.1" 200 None 2026-03-24 07:59:53,392 - lava_callback - ERROR - Error processing callback for node 69bfa168f1004e82292e499a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39862 - "GET / HTTP/1.0" 200 OK 2026-03-24 07:59:57,558 - lava_callback - INFO - Received callback for node 69c17ec42b2b51e9ad97a959 INFO: 172.21.0.1:39872 - "POST /node/69c17ec42b2b51e9ad97a959 HTTP/1.0" 202 Accepted 2026-03-24 07:59:57,565 - lava_callback - INFO - Starting async job submit for node 69c17ec42b2b51e9ad97a959 2026-03-24 07:59:57,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:59:57,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c17ec42b2b51e9ad97a959 HTTP/1.1" 200 None 2026-03-24 07:59:57,603 - lava_callback - DEBUG - Processing node 69c17ec42b2b51e9ad97a959: baseline-arm64-broonie 2026-03-24 07:59:57,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 07:59:58,120 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 07:59:58,122 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c17ec42b2b51e9ad97a959/log.txt.gz 2026-03-24 07:59:58,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 07:59:58,527 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 07:59:58,529 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c17ec42b2b51e9ad97a959/lava-logs.yaml 2026-03-24 07:59:58,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 07:59:58,700 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 07:59:58,702 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c17ec42b2b51e9ad97a959/lava_callback.json.gz 2026-03-24 07:59:58,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:59:58,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c17ec42b2b51e9ad97a959 HTTP/1.1" 200 None 2026-03-24 07:59:58,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:59:58,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-24 07:59:58,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 07:59:58,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c17ec42b2b51e9ad97a959 HTTP/1.1" 200 None 2026-03-24 07:59:58,790 - lava_callback - INFO - Completed processing callback for node 69c17ec42b2b51e9ad97a959 INFO: 172.21.0.1:47096 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:00:11,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:00:11,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:43528 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54622 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:01:13,329 - lava_callback - INFO - Received callback for node 69bfa1b5f1004e82292e49ac INFO: 172.21.0.1:49356 - "POST /node/69bfa1b5f1004e82292e49ac HTTP/1.0" 202 Accepted 2026-03-24 08:01:13,335 - lava_callback - INFO - Starting async job submit for node 69bfa1b5f1004e82292e49ac 2026-03-24 08:01:13,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:01:13,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa1b5f1004e82292e49ac HTTP/1.1" 200 None 2026-03-24 08:01:13,365 - lava_callback - DEBUG - Processing node 69bfa1b5f1004e82292e49ac: baseline-arm64-clabbe 2026-03-24 08:01:13,365 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfa1b5f1004e82292e49ac 2026-03-24 08:01:13,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:01:13,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa1b5f1004e82292e49ac HTTP/1.1" 200 None 2026-03-24 08:01:13,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:01:13,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21b717e5cb936602bc2b HTTP/1.1" 200 None 2026-03-24 08:01:13,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:01:13,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfa1b5f1004e82292e49ac HTTP/1.1" 200 None 2026-03-24 08:01:13,428 - lava_callback - ERROR - Error processing callback for node 69bfa1b5f1004e82292e49ac: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49358 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49720 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:02:13,390 - lava_callback - INFO - Received callback for node 69bfa201f1004e82292e49c2 INFO: 172.21.0.1:47470 - "POST /node/69bfa201f1004e82292e49c2 HTTP/1.0" 202 Accepted 2026-03-24 08:02:13,396 - lava_callback - INFO - Starting async job submit for node 69bfa201f1004e82292e49c2 2026-03-24 08:02:13,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:02:13,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa201f1004e82292e49c2 HTTP/1.1" 200 None 2026-03-24 08:02:13,412 - lava_callback - DEBUG - Processing node 69bfa201f1004e82292e49c2: baseline-arm64-clabbe 2026-03-24 08:02:13,412 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfa201f1004e82292e49c2 2026-03-24 08:02:13,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:02:13,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa201f1004e82292e49c2 HTTP/1.1" 200 None 2026-03-24 08:02:13,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:02:13,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21b717e5cb936602bc2b HTTP/1.1" 200 None 2026-03-24 08:02:13,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:02:13,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfa201f1004e82292e49c2 HTTP/1.1" 200 None 2026-03-24 08:02:13,467 - lava_callback - ERROR - Error processing callback for node 69bfa201f1004e82292e49c2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47482 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35616 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43790 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:03:33,469 - lava_callback - INFO - Received callback for node 69bfa24cf1004e82292e49cb INFO: 172.21.0.1:55108 - "POST /node/69bfa24cf1004e82292e49cb HTTP/1.0" 202 Accepted 2026-03-24 08:03:33,475 - lava_callback - INFO - Starting async job submit for node 69bfa24cf1004e82292e49cb 2026-03-24 08:03:33,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:03:33,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa24cf1004e82292e49cb HTTP/1.1" 200 None 2026-03-24 08:03:33,508 - lava_callback - DEBUG - Processing node 69bfa24cf1004e82292e49cb: baseline-arm64-clabbe 2026-03-24 08:03:33,508 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfa24cf1004e82292e49cb 2026-03-24 08:03:33,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:03:33,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa24cf1004e82292e49cb HTTP/1.1" 200 None 2026-03-24 08:03:33,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:03:33,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21b717e5cb936602bc2b HTTP/1.1" 200 None 2026-03-24 08:03:33,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:03:33,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfa24cf1004e82292e49cb HTTP/1.1" 200 None 2026-03-24 08:03:33,574 - lava_callback - ERROR - Error processing callback for node 69bfa24cf1004e82292e49cb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56486 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55348 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59848 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:05:53,601 - lava_callback - INFO - Received callback for node 69bfa2dff1004e82292e49e9 INFO: 172.21.0.1:46626 - "POST /node/69bfa2dff1004e82292e49e9 HTTP/1.0" 202 Accepted 2026-03-24 08:05:53,607 - lava_callback - INFO - Starting async job submit for node 69bfa2dff1004e82292e49e9 2026-03-24 08:05:53,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:05:53,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa2dff1004e82292e49e9 HTTP/1.1" 200 None 2026-03-24 08:05:53,625 - lava_callback - DEBUG - Processing node 69bfa2dff1004e82292e49e9: baseline-arm64-clabbe 2026-03-24 08:05:53,626 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfa2dff1004e82292e49e9 2026-03-24 08:05:53,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:05:53,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa2dff1004e82292e49e9 HTTP/1.1" 200 None 2026-03-24 08:05:53,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:05:53,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21b717e5cb936602bc2b HTTP/1.1" 200 None 2026-03-24 08:05:53,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:05:53,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfa2dff1004e82292e49e9 HTTP/1.1" 200 None 2026-03-24 08:05:53,682 - lava_callback - ERROR - Error processing callback for node 69bfa2dff1004e82292e49e9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34114 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41870 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42250 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42276 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:08:17,044 - lava_callback - INFO - Received callback for node 69bfa36df1004e82292e4a02 INFO: 172.21.0.1:42292 - "POST /node/69bfa36df1004e82292e4a02 HTTP/1.0" 202 Accepted 2026-03-24 08:08:17,049 - lava_callback - INFO - Starting async job submit for node 69bfa36df1004e82292e4a02 2026-03-24 08:08:17,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:08:17,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa36df1004e82292e4a02 HTTP/1.1" 200 None 2026-03-24 08:08:17,068 - lava_callback - DEBUG - Processing node 69bfa36df1004e82292e4a02: baseline-arm64-baylibre 2026-03-24 08:08:17,068 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfa36df1004e82292e4a02 2026-03-24 08:08:17,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:08:17,082 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa36df1004e82292e4a02 HTTP/1.1" 200 None 2026-03-24 08:08:17,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:08:17,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-24 08:08:17,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:08:17,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfa36df1004e82292e4a02 HTTP/1.1" 200 None 2026-03-24 08:08:17,122 - lava_callback - ERROR - Error processing callback for node 69bfa36df1004e82292e4a02: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46710 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44164 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60934 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:10:53,725 - lava_callback - INFO - Received callback for node 69bfa406f1004e82292e4a1e INFO: 172.21.0.1:48130 - "POST /node/69bfa406f1004e82292e4a1e HTTP/1.0" 202 Accepted 2026-03-24 08:10:53,731 - lava_callback - INFO - Starting async job submit for node 69bfa406f1004e82292e4a1e 2026-03-24 08:10:53,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:10:53,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa406f1004e82292e4a1e HTTP/1.1" 200 None 2026-03-24 08:10:53,756 - lava_callback - DEBUG - Processing node 69bfa406f1004e82292e4a1e: baseline-arm64-clabbe 2026-03-24 08:10:53,756 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfa406f1004e82292e4a1e 2026-03-24 08:10:53,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:10:53,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa406f1004e82292e4a1e HTTP/1.1" 200 None 2026-03-24 08:10:53,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:10:53,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-24 08:10:53,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:10:53,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfa406f1004e82292e4a1e HTTP/1.1" 200 None 2026-03-24 08:10:53,817 - lava_callback - ERROR - Error processing callback for node 69bfa406f1004e82292e4a1e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39628 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53870 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:12:13,816 - lava_callback - INFO - Received callback for node 69bfa452f1004e82292e4a25 INFO: 172.21.0.1:50020 - "POST /node/69bfa452f1004e82292e4a25 HTTP/1.0" 202 Accepted 2026-03-24 08:12:13,821 - lava_callback - INFO - Starting async job submit for node 69bfa452f1004e82292e4a25 2026-03-24 08:12:13,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:12:13,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa452f1004e82292e4a25 HTTP/1.1" 200 None 2026-03-24 08:12:13,837 - lava_callback - DEBUG - Processing node 69bfa452f1004e82292e4a25: baseline-arm64-clabbe 2026-03-24 08:12:13,838 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfa452f1004e82292e4a25 2026-03-24 08:12:13,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:12:13,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa452f1004e82292e4a25 HTTP/1.1" 200 None 2026-03-24 08:12:13,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:12:13,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-24 08:12:13,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:12:13,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfa452f1004e82292e4a25 HTTP/1.1" 200 None 2026-03-24 08:12:13,901 - lava_callback - ERROR - Error processing callback for node 69bfa452f1004e82292e4a25: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50028 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35542 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32956 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:13:53,938 - lava_callback - INFO - Received callback for node 69bfa4ae7544ab5cd4a7d9a4 INFO: 172.21.0.1:60682 - "POST /node/69bfa4ae7544ab5cd4a7d9a4 HTTP/1.0" 202 Accepted 2026-03-24 08:13:53,946 - lava_callback - INFO - Starting async job submit for node 69bfa4ae7544ab5cd4a7d9a4 2026-03-24 08:13:53,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:13:53,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa4ae7544ab5cd4a7d9a4 HTTP/1.1" 200 None 2026-03-24 08:13:53,971 - lava_callback - DEBUG - Processing node 69bfa4ae7544ab5cd4a7d9a4: baseline-arm64-clabbe 2026-03-24 08:13:53,971 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfa4ae7544ab5cd4a7d9a4 2026-03-24 08:13:53,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:13:53,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa4ae7544ab5cd4a7d9a4 HTTP/1.1" 200 None 2026-03-24 08:13:54,000 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:13:54,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21b717e5cb936602bc2b HTTP/1.1" 200 None 2026-03-24 08:13:54,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:13:54,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfa4ae7544ab5cd4a7d9a4 HTTP/1.1" 200 None 2026-03-24 08:13:54,060 - lava_callback - ERROR - Error processing callback for node 69bfa4ae7544ab5cd4a7d9a4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33812 - "GET / HTTP/1.0" 200 OK INFO: Shutting down INFO: Waiting for application shutdown. INFO: Application shutdown complete. INFO: Finished server process [1]