INFO: Started server process [1] INFO: Waiting for application startup. INFO: Application startup complete. INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit) INFO: 172.20.0.1:59504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39250 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53960 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51616 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34132 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55400 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41486 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57328 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56664 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52492 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34376 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38426 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48448 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49862 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56732 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53612 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55374 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50718 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50730 - "GET / HTTP/1.0" 200 OK 2026-03-10 00:31:20,173 - lava_callback - INFO - Received callback for node 69af65d1275c8d3b79399df8 2026-03-10 00:31:20,180 - lava_callback - INFO - Starting async job submit for node 69af65d1275c8d3b79399df8 INFO: 172.20.0.1:49324 - "POST /node/69af65d1275c8d3b79399df8 HTTP/1.0" 202 Accepted 2026-03-10 00:31:20,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:31:20,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af65d1275c8d3b79399df8 HTTP/1.1" 200 None 2026-03-10 00:31:20,224 - lava_callback - DEBUG - Processing node 69af65d1275c8d3b79399df8: baseline-arm 2026-03-10 00:31:20,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:31:20,577 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:31:20,579 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69af65d1275c8d3b79399df8/log.txt.gz 2026-03-10 00:31:20,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:31:20,899 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:31:20,901 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69af65d1275c8d3b79399df8/lava-logs.yaml 2026-03-10 00:31:20,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:31:21,063 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:31:21,065 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69af65d1275c8d3b79399df8/lava_callback.json.gz 2026-03-10 00:31:21,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:31:21,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af65d1275c8d3b79399df8 HTTP/1.1" 200 None 2026-03-10 00:31:21,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:31:21,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6334275c8d3b79399dba HTTP/1.1" 200 None 2026-03-10 00:31:21,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:31:21,170 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af65d1275c8d3b79399df8 HTTP/1.1" 200 None 2026-03-10 00:31:21,177 - lava_callback - INFO - Completed processing callback for node 69af65d1275c8d3b79399df8 INFO: 172.20.0.1:49326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52334 - "GET / HTTP/1.0" 200 OK 2026-03-10 00:31:51,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:31:51,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:42410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35034 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39840 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41154 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57846 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34500 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34516 - "GET /node/6991fb1b01ff1e8e0859ea8d%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:34524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43156 - "GET /node/698c5088755d7811a41414da%5Cn HTTP/1.0" 405 Method Not Allowed 2026-03-10 00:36:34,925 - lava_callback - INFO - Received callback for node 69af6787275c8d3b79399e16 INFO: 172.20.0.1:43166 - "POST /node/69af6787275c8d3b79399e16 HTTP/1.0" 202 Accepted 2026-03-10 00:36:34,935 - lava_callback - INFO - Starting async job submit for node 69af6787275c8d3b79399e16 2026-03-10 00:36:34,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:36:34,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6787275c8d3b79399e16 HTTP/1.1" 200 None 2026-03-10 00:36:34,978 - lava_callback - DEBUG - Processing node 69af6787275c8d3b79399e16: baseline-arm 2026-03-10 00:36:34,987 - lava_callback - WARNING - Failed to upload log 2026-03-10 00:36:34,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:36:35,108 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:36:35,110 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69af6787275c8d3b79399e16/lava-logs.yaml 2026-03-10 00:36:35,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:36:35,230 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:36:35,232 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69af6787275c8d3b79399e16/lava_callback.json.gz 2026-03-10 00:36:35,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:36:35,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6787275c8d3b79399e16 HTTP/1.1" 200 None 2026-03-10 00:36:35,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:36:35,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af632c275c8d3b79399db4 HTTP/1.1" 200 None 2026-03-10 00:36:35,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:36:35,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af6787275c8d3b79399e16 HTTP/1.1" 200 None 2026-03-10 00:36:35,306 - lava_callback - INFO - Completed processing callback for node 69af6787275c8d3b79399e16 INFO: 172.20.0.1:43172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52144 - "GET / HTTP/1.0" 200 OK 2026-03-10 00:36:51,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:36:51,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:55982 - "GET / HTTP/1.0" 200 OK 2026-03-10 00:36:59,046 - lava_callback - INFO - Received callback for node 69af6794275c8d3b79399e38 INFO: 172.20.0.1:55998 - "POST /node/69af6794275c8d3b79399e38 HTTP/1.0" 202 Accepted 2026-03-10 00:36:59,053 - lava_callback - INFO - Starting async job submit for node 69af6794275c8d3b79399e38 2026-03-10 00:36:59,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:36:59,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6794275c8d3b79399e38 HTTP/1.1" 200 None 2026-03-10 00:36:59,072 - lava_callback - DEBUG - Processing node 69af6794275c8d3b79399e38: baseline-arm 2026-03-10 00:36:59,077 - lava_callback - WARNING - Failed to upload log 2026-03-10 00:36:59,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:36:59,217 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:36:59,219 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69af6794275c8d3b79399e38/lava-logs.yaml 2026-03-10 00:36:59,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:36:59,365 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:36:59,366 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69af6794275c8d3b79399e38/lava_callback.json.gz 2026-03-10 00:36:59,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:36:59,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6794275c8d3b79399e38 HTTP/1.1" 200 None 2026-03-10 00:36:59,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:36:59,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af632c275c8d3b79399db4 HTTP/1.1" 200 None 2026-03-10 00:36:59,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:36:59,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af6794275c8d3b79399e38 HTTP/1.1" 200 None 2026-03-10 00:36:59,426 - lava_callback - INFO - Completed processing callback for node 69af6794275c8d3b79399e38 2026-03-10 00:37:01,187 - lava_callback - INFO - Received callback for node 69af6788275c8d3b79399e19 INFO: 172.20.0.1:35444 - "POST /node/69af6788275c8d3b79399e19 HTTP/1.0" 202 Accepted 2026-03-10 00:37:01,233 - lava_callback - INFO - Starting async job submit for node 69af6788275c8d3b79399e19 2026-03-10 00:37:01,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:37:01,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6788275c8d3b79399e19 HTTP/1.1" 200 None 2026-03-10 00:37:01,266 - lava_callback - DEBUG - Processing node 69af6788275c8d3b79399e19: baseline-arm 2026-03-10 00:37:01,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:37:01,511 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:37:01,513 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69af6788275c8d3b79399e19/log.txt.gz 2026-03-10 00:37:01,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:37:01,785 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:37:01,787 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69af6788275c8d3b79399e19/lava-logs.yaml 2026-03-10 00:37:01,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:37:01,900 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:37:01,902 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69af6788275c8d3b79399e19/lava_callback.json.gz 2026-03-10 00:37:01,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:37:01,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6788275c8d3b79399e19 HTTP/1.1" 200 None 2026-03-10 00:37:01,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:37:01,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af632c275c8d3b79399db4 HTTP/1.1" 200 None 2026-03-10 00:37:01,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:37:01,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af6788275c8d3b79399e19 HTTP/1.1" 200 None 2026-03-10 00:37:01,974 - lava_callback - INFO - Completed processing callback for node 69af6788275c8d3b79399e19 INFO: 172.20.0.1:35454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33244 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33256 - "GET / HTTP/1.0" 200 OK 2026-03-10 00:37:20,841 - lava_callback - INFO - Received callback for node 69af678b275c8d3b79399e23 INFO: 172.20.0.1:36844 - "POST /node/69af678b275c8d3b79399e23 HTTP/1.0" 202 Accepted 2026-03-10 00:37:20,889 - lava_callback - INFO - Starting async job submit for node 69af678b275c8d3b79399e23 2026-03-10 00:37:20,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:37:20,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af678b275c8d3b79399e23 HTTP/1.1" 200 None 2026-03-10 00:37:20,922 - lava_callback - DEBUG - Processing node 69af678b275c8d3b79399e23: baseline-arm 2026-03-10 00:37:21,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:37:21,184 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:37:21,186 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69af678b275c8d3b79399e23/log.txt.gz 2026-03-10 00:37:21,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:37:21,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:37:21,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-10 00:37:21,411 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:37:21,413 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69af678b275c8d3b79399e23/lava-logs.yaml 2026-03-10 00:37:21,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:37:21,553 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:37:21,555 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69af678b275c8d3b79399e23/lava_callback.json.gz 2026-03-10 00:37:21,573 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:37:21,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af678b275c8d3b79399e23 HTTP/1.1" 200 None 2026-03-10 00:37:21,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:37:21,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af632c275c8d3b79399db4 HTTP/1.1" 200 None 2026-03-10 00:37:21,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:37:21,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af678b275c8d3b79399e23 HTTP/1.1" 200 None 2026-03-10 00:37:21,652 - lava_callback - INFO - Completed processing callback for node 69af678b275c8d3b79399e23 INFO: 172.20.0.1:36846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51352 - "GET / HTTP/1.0" 200 OK 2026-03-10 00:37:39,126 - lava_callback - INFO - Received callback for node 69af678a275c8d3b79399e1f INFO: 172.20.0.1:51360 - "POST /node/69af678a275c8d3b79399e1f HTTP/1.0" 202 Accepted 2026-03-10 00:37:39,133 - lava_callback - INFO - Starting async job submit for node 69af678a275c8d3b79399e1f 2026-03-10 00:37:39,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:37:39,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af678a275c8d3b79399e1f HTTP/1.1" 200 None 2026-03-10 00:37:39,179 - lava_callback - DEBUG - Processing node 69af678a275c8d3b79399e1f: baseline-arm 2026-03-10 00:37:39,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:37:39,594 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:37:39,595 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69af678a275c8d3b79399e1f/log.txt.gz 2026-03-10 00:37:39,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:37:39,932 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:37:39,934 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69af678a275c8d3b79399e1f/lava-logs.yaml 2026-03-10 00:37:39,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:37:40,128 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:37:40,129 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69af678a275c8d3b79399e1f/lava_callback.json.gz 2026-03-10 00:37:40,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:37:40,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af678a275c8d3b79399e1f HTTP/1.1" 200 None 2026-03-10 00:37:40,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:37:40,183 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af632c275c8d3b79399db4 HTTP/1.1" 200 None 2026-03-10 00:37:40,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:37:40,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af678a275c8d3b79399e1f HTTP/1.1" 200 None 2026-03-10 00:37:40,214 - lava_callback - INFO - Completed processing callback for node 69af678a275c8d3b79399e1f INFO: 172.20.0.1:53574 - "GET / HTTP/1.0" 200 OK 2026-03-10 00:37:48,066 - lava_callback - INFO - Received callback for node 69af6787275c8d3b79399e13 INFO: 172.20.0.1:53578 - "POST /node/69af6787275c8d3b79399e13 HTTP/1.0" 202 Accepted 2026-03-10 00:37:48,072 - lava_callback - INFO - Starting async job submit for node 69af6787275c8d3b79399e13 2026-03-10 00:37:48,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:37:48,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6787275c8d3b79399e13 HTTP/1.1" 200 None 2026-03-10 00:37:48,108 - lava_callback - DEBUG - Processing node 69af6787275c8d3b79399e13: baseline-arm 2026-03-10 00:37:48,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:37:48,520 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:37:48,522 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69af6787275c8d3b79399e13/log.txt.gz 2026-03-10 00:37:48,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:37:48,813 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:37:48,815 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69af6787275c8d3b79399e13/lava-logs.yaml 2026-03-10 00:37:48,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:37:48,957 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:37:48,958 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69af6787275c8d3b79399e13/lava_callback.json.gz 2026-03-10 00:37:48,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:37:48,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6787275c8d3b79399e13 HTTP/1.1" 200 None 2026-03-10 00:37:48,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:37:49,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af632c275c8d3b79399db4 HTTP/1.1" 200 None 2026-03-10 00:37:49,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:37:49,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af6787275c8d3b79399e13 HTTP/1.1" 200 None 2026-03-10 00:37:49,046 - lava_callback - INFO - Completed processing callback for node 69af6787275c8d3b79399e13 2026-03-10 00:37:51,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:37:51,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:47958 - "GET / HTTP/1.0" 200 OK 2026-03-10 00:38:00,716 - lava_callback - INFO - Received callback for node 69af67ac275c8d3b79399e9f INFO: 172.20.0.1:32970 - "POST /node/69af67ac275c8d3b79399e9f HTTP/1.0" 202 Accepted 2026-03-10 00:38:00,722 - lava_callback - INFO - Starting async job submit for node 69af67ac275c8d3b79399e9f 2026-03-10 00:38:00,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:38:00,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af67ac275c8d3b79399e9f HTTP/1.1" 200 None 2026-03-10 00:38:00,756 - lava_callback - DEBUG - Processing node 69af67ac275c8d3b79399e9f: baseline-arm-pengutronix 2026-03-10 00:38:00,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:38:01,014 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:38:01,016 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69af67ac275c8d3b79399e9f/log.txt.gz 2026-03-10 00:38:01,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:38:01,269 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:38:01,271 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69af67ac275c8d3b79399e9f/lava-logs.yaml 2026-03-10 00:38:01,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:38:01,408 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:38:01,410 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69af67ac275c8d3b79399e9f/lava_callback.json.gz 2026-03-10 00:38:01,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:38:01,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af67ac275c8d3b79399e9f HTTP/1.1" 200 None 2026-03-10 00:38:01,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:38:01,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af632c275c8d3b79399db4 HTTP/1.1" 200 None 2026-03-10 00:38:01,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:38:01,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af67ac275c8d3b79399e9f HTTP/1.1" 200 None 2026-03-10 00:38:01,504 - lava_callback - INFO - Completed processing callback for node 69af67ac275c8d3b79399e9f INFO: 172.20.0.1:32984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32998 - "GET / HTTP/1.0" 200 OK 2026-03-10 00:38:11,213 - lava_callback - INFO - Received callback for node 69af67b0275c8d3b79399eac INFO: 172.20.0.1:34864 - "POST /node/69af67b0275c8d3b79399eac HTTP/1.0" 202 Accepted 2026-03-10 00:38:11,261 - lava_callback - INFO - Starting async job submit for node 69af67b0275c8d3b79399eac 2026-03-10 00:38:11,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:38:11,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af67b0275c8d3b79399eac HTTP/1.1" 200 None 2026-03-10 00:38:11,297 - lava_callback - DEBUG - Processing node 69af67b0275c8d3b79399eac: baseline-arm-pengutronix 2026-03-10 00:38:11,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:38:11,513 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:38:11,514 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69af67b0275c8d3b79399eac/log.txt.gz 2026-03-10 00:38:11,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:38:11,721 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:38:11,723 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69af67b0275c8d3b79399eac/lava-logs.yaml 2026-03-10 00:38:11,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:38:11,844 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:38:11,846 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69af67b0275c8d3b79399eac/lava_callback.json.gz 2026-03-10 00:38:11,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:38:11,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af67b0275c8d3b79399eac HTTP/1.1" 200 None 2026-03-10 00:38:11,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:38:11,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af632c275c8d3b79399db4 HTTP/1.1" 200 None 2026-03-10 00:38:11,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:38:11,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af67b0275c8d3b79399eac HTTP/1.1" 200 None 2026-03-10 00:38:11,937 - lava_callback - INFO - Completed processing callback for node 69af67b0275c8d3b79399eac INFO: 172.20.0.1:34868 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-10 00:38:15,693 - lava_callback - INFO - Received callback for node 69af67a1275c8d3b79399e78 INFO: 172.20.0.1:34876 - "POST /node/69af67a1275c8d3b79399e78 HTTP/1.0" 202 Accepted 2026-03-10 00:38:15,699 - lava_callback - INFO - Starting async job submit for node 69af67a1275c8d3b79399e78 2026-03-10 00:38:15,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:38:15,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af67a1275c8d3b79399e78 HTTP/1.1" 200 None 2026-03-10 00:38:15,736 - lava_callback - DEBUG - Processing node 69af67a1275c8d3b79399e78: baseline-arm-clabbe 2026-03-10 00:38:15,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:38:16,075 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:38:16,076 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69af67a1275c8d3b79399e78/log.txt.gz 2026-03-10 00:38:16,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:38:16,386 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:38:16,388 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69af67a1275c8d3b79399e78/lava-logs.yaml 2026-03-10 00:38:16,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:34880 - "GET / HTTP/1.0" 200 OK 2026-03-10 00:38:16,539 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:38:16,541 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69af67a1275c8d3b79399e78/lava_callback.json.gz 2026-03-10 00:38:16,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:38:16,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af67a1275c8d3b79399e78 HTTP/1.1" 200 None 2026-03-10 00:38:16,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:38:16,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af632c275c8d3b79399db4 HTTP/1.1" 200 None 2026-03-10 00:38:16,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:38:16,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af67a1275c8d3b79399e78 HTTP/1.1" 200 None 2026-03-10 00:38:16,624 - lava_callback - INFO - Completed processing callback for node 69af67a1275c8d3b79399e78 2026-03-10 00:38:18,135 - lava_callback - INFO - Received callback for node 69af67ae275c8d3b79399ea6 INFO: 172.20.0.1:34894 - "POST /node/69af67ae275c8d3b79399ea6 HTTP/1.0" 202 Accepted 2026-03-10 00:38:18,141 - lava_callback - INFO - Starting async job submit for node 69af67ae275c8d3b79399ea6 2026-03-10 00:38:18,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:38:18,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af67ae275c8d3b79399ea6 HTTP/1.1" 200 None 2026-03-10 00:38:18,179 - lava_callback - DEBUG - Processing node 69af67ae275c8d3b79399ea6: baseline-arm-pengutronix 2026-03-10 00:38:18,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:38:18,458 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:38:18,460 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69af67ae275c8d3b79399ea6/log.txt.gz 2026-03-10 00:38:18,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:38:18,723 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:38:18,725 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69af67ae275c8d3b79399ea6/lava-logs.yaml 2026-03-10 00:38:18,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:38:18,860 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:38:18,861 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69af67ae275c8d3b79399ea6/lava_callback.json.gz 2026-03-10 00:38:18,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:38:18,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af67ae275c8d3b79399ea6 HTTP/1.1" 200 None 2026-03-10 00:38:18,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:38:18,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af632c275c8d3b79399db4 HTTP/1.1" 200 None 2026-03-10 00:38:18,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:38:18,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af67ae275c8d3b79399ea6 HTTP/1.1" 200 None 2026-03-10 00:38:18,937 - lava_callback - INFO - Completed processing callback for node 69af67ae275c8d3b79399ea6 2026-03-10 00:38:19,715 - lava_callback - INFO - Received callback for node 69af679b275c8d3b79399e4b INFO: 172.20.0.1:35630 - "POST /node/69af679b275c8d3b79399e4b HTTP/1.0" 202 Accepted 2026-03-10 00:38:19,721 - lava_callback - INFO - Starting async job submit for node 69af679b275c8d3b79399e4b 2026-03-10 00:38:19,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:38:19,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af679b275c8d3b79399e4b HTTP/1.1" 200 None 2026-03-10 00:38:19,759 - lava_callback - DEBUG - Processing node 69af679b275c8d3b79399e4b: baseline-arm-broonie 2026-03-10 00:38:19,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:38:20,072 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:38:20,073 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69af679b275c8d3b79399e4b/log.txt.gz 2026-03-10 00:38:20,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:38:20,366 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:38:20,369 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69af679b275c8d3b79399e4b/lava-logs.yaml 2026-03-10 00:38:20,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:38:20,507 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:38:20,509 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69af679b275c8d3b79399e4b/lava_callback.json.gz 2026-03-10 00:38:20,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:38:20,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af679b275c8d3b79399e4b HTTP/1.1" 200 None 2026-03-10 00:38:20,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:38:20,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af632c275c8d3b79399db4 HTTP/1.1" 200 None 2026-03-10 00:38:20,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:38:20,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af679b275c8d3b79399e4b HTTP/1.1" 200 None 2026-03-10 00:38:20,591 - lava_callback - INFO - Completed processing callback for node 69af679b275c8d3b79399e4b 2026-03-10 00:38:21,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:38:21,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-10 00:38:26,256 - lava_callback - INFO - Received callback for node 69af67aa275c8d3b79399e92 INFO: 172.20.0.1:35644 - "POST /node/69af67aa275c8d3b79399e92 HTTP/1.0" 202 Accepted 2026-03-10 00:38:26,262 - lava_callback - INFO - Starting async job submit for node 69af67aa275c8d3b79399e92 2026-03-10 00:38:26,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:38:26,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af67aa275c8d3b79399e92 HTTP/1.1" 200 None 2026-03-10 00:38:26,313 - lava_callback - DEBUG - Processing node 69af67aa275c8d3b79399e92: baseline-arm-kontron 2026-03-10 00:38:26,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:35654 - "GET / HTTP/1.0" 200 OK 2026-03-10 00:38:26,643 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:38:26,645 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-69af67aa275c8d3b79399e92/log.txt.gz 2026-03-10 00:38:26,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:38:26,931 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:38:26,933 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-69af67aa275c8d3b79399e92/lava-logs.yaml 2026-03-10 00:38:26,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:38:27,099 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:38:27,101 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-69af67aa275c8d3b79399e92/lava_callback.json.gz 2026-03-10 00:38:27,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:38:27,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af67aa275c8d3b79399e92 HTTP/1.1" 200 None 2026-03-10 00:38:27,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:38:27,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af632c275c8d3b79399db4 HTTP/1.1" 200 None 2026-03-10 00:38:27,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:38:27,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af67aa275c8d3b79399e92 HTTP/1.1" 200 None 2026-03-10 00:38:27,189 - lava_callback - INFO - Completed processing callback for node 69af67aa275c8d3b79399e92 INFO: 172.20.0.1:51248 - "GET / HTTP/1.0" 200 OK 2026-03-10 00:38:38,225 - lava_callback - INFO - Received callback for node 69acc4ffbcd207e915d76cda INFO: 172.20.0.1:51260 - "POST /node/69acc4ffbcd207e915d76cda HTTP/1.0" 202 Accepted 2026-03-10 00:38:38,230 - lava_callback - INFO - Starting async job submit for node 69acc4ffbcd207e915d76cda 2026-03-10 00:38:38,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:38:38,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acc4ffbcd207e915d76cda HTTP/1.1" 200 None 2026-03-10 00:38:38,246 - lava_callback - DEBUG - Processing node 69acc4ffbcd207e915d76cda: baseline-arm-clabbe 2026-03-10 00:38:38,246 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acc4ffbcd207e915d76cda 2026-03-10 00:38:38,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:38:38,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acc4ffbcd207e915d76cda HTTP/1.1" 200 None 2026-03-10 00:38:38,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:38:38,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa1d368c4d487a0e751145 HTTP/1.1" 200 None 2026-03-10 00:38:38,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:38:38,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acc4ffbcd207e915d76cda HTTP/1.1" 200 None 2026-03-10 00:38:38,315 - lava_callback - ERROR - Error processing callback for node 69acc4ffbcd207e915d76cda: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-10 00:38:39,624 - lava_callback - INFO - Received callback for node 69af679a275c8d3b79399e48 INFO: 172.20.0.1:51262 - "POST /node/69af679a275c8d3b79399e48 HTTP/1.0" 202 Accepted 2026-03-10 00:38:39,630 - lava_callback - INFO - Starting async job submit for node 69af679a275c8d3b79399e48 2026-03-10 00:38:39,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:38:39,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af679a275c8d3b79399e48 HTTP/1.1" 200 None 2026-03-10 00:38:39,668 - lava_callback - DEBUG - Processing node 69af679a275c8d3b79399e48: baseline-arm-broonie 2026-03-10 00:38:39,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:38:40,010 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:38:40,011 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69af679a275c8d3b79399e48/log.txt.gz 2026-03-10 00:38:40,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:38:40,343 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:38:40,345 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69af679a275c8d3b79399e48/lava-logs.yaml 2026-03-10 00:38:40,351 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:38:40,506 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:38:40,508 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69af679a275c8d3b79399e48/lava_callback.json.gz 2026-03-10 00:38:40,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:38:40,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af679a275c8d3b79399e48 HTTP/1.1" 200 None 2026-03-10 00:38:40,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:38:40,563 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af632c275c8d3b79399db4 HTTP/1.1" 200 None 2026-03-10 00:38:40,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:38:40,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af679a275c8d3b79399e48 HTTP/1.1" 200 None 2026-03-10 00:38:40,589 - lava_callback - INFO - Completed processing callback for node 69af679a275c8d3b79399e48 INFO: 172.20.0.1:36626 - "GET / HTTP/1.0" 200 OK 2026-03-10 00:38:51,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:38:51,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:49254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53276 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53278 - "GET / HTTP/1.0" 200 OK 2026-03-10 00:39:18,526 - lava_callback - INFO - Received callback for node 69af67ca275c8d3b79399ee5 INFO: 172.20.0.1:53282 - "POST /node/69af67ca275c8d3b79399ee5 HTTP/1.0" 202 Accepted 2026-03-10 00:39:18,537 - lava_callback - INFO - Starting async job submit for node 69af67ca275c8d3b79399ee5 2026-03-10 00:39:18,570 - lava_callback - INFO - Received callback for node 69acc525bcd207e915d76cfb 2026-03-10 00:39:18,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:39:18,575 - lava_callback - INFO - Starting async job submit for node 69acc525bcd207e915d76cfb INFO: 172.20.0.1:53290 - "POST /node/69acc525bcd207e915d76cfb HTTP/1.0" 202 Accepted 2026-03-10 00:39:18,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:39:18,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af67ca275c8d3b79399ee5 HTTP/1.1" 200 None 2026-03-10 00:39:18,598 - lava_callback - DEBUG - Processing node 69af67ca275c8d3b79399ee5: ltp-dio 2026-03-10 00:39:18,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acc525bcd207e915d76cfb HTTP/1.1" 200 None 2026-03-10 00:39:18,621 - lava_callback - DEBUG - Processing node 69acc525bcd207e915d76cfb: baseline-arm-clabbe 2026-03-10 00:39:18,621 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acc525bcd207e915d76cfb 2026-03-10 00:39:18,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:39:18,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acc525bcd207e915d76cfb HTTP/1.1" 200 None 2026-03-10 00:39:18,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:39:18,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa1d368c4d487a0e751145 HTTP/1.1" 200 None 2026-03-10 00:39:18,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:39:18,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acc525bcd207e915d76cfb HTTP/1.1" 200 None 2026-03-10 00:39:18,972 - lava_callback - ERROR - Error processing callback for node 69acc525bcd207e915d76cfb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-10 00:39:19,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:39:19,358 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:39:19,361 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-dio-69af67ca275c8d3b79399ee5/log.txt.gz 2026-03-10 00:39:19,599 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:39:19,781 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:39:19,783 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-dio-69af67ca275c8d3b79399ee5/lava-logs.yaml 2026-03-10 00:39:19,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:39:19,950 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:39:19,952 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-dio-69af67ca275c8d3b79399ee5/lava_callback.json.gz 2026-03-10 00:39:19,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:39:19,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af67ca275c8d3b79399ee5 HTTP/1.1" 200 None 2026-03-10 00:39:19,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:39:20,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af632c275c8d3b79399db4 HTTP/1.1" 200 None 2026-03-10 00:39:20,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:39:20,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af67ca275c8d3b79399ee5 HTTP/1.1" 200 None 2026-03-10 00:39:20,044 - lava_callback - INFO - Completed processing callback for node 69af67ca275c8d3b79399ee5 2026-03-10 00:39:21,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:39:21,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:36186 - "GET / HTTP/1.0" 200 OK 2026-03-10 00:39:33,821 - lava_callback - INFO - Received callback for node 69af6836275c8d3b79399f7a INFO: 172.20.0.1:40056 - "POST /node/69af6836275c8d3b79399f7a HTTP/1.0" 202 Accepted 2026-03-10 00:39:33,827 - lava_callback - INFO - Starting async job submit for node 69af6836275c8d3b79399f7a 2026-03-10 00:39:33,830 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:39:33,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6836275c8d3b79399f7a HTTP/1.1" 200 None 2026-03-10 00:39:33,861 - lava_callback - DEBUG - Processing node 69af6836275c8d3b79399f7a: baseline-arm 2026-03-10 00:39:33,867 - lava_callback - WARNING - Failed to upload log 2026-03-10 00:39:33,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:39:34,016 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:39:34,018 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69af6836275c8d3b79399f7a/lava-logs.yaml 2026-03-10 00:39:34,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:39:34,161 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:39:34,162 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69af6836275c8d3b79399f7a/lava_callback.json.gz 2026-03-10 00:39:34,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:39:34,180 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6836275c8d3b79399f7a HTTP/1.1" 200 None 2026-03-10 00:39:34,181 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:39:34,203 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af632c275c8d3b79399db4 HTTP/1.1" 200 None 2026-03-10 00:39:34,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:39:34,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af6836275c8d3b79399f7a HTTP/1.1" 200 None 2026-03-10 00:39:34,252 - lava_callback - INFO - Completed processing callback for node 69af6836275c8d3b79399f7a INFO: 172.20.0.1:40062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54254 - "GET / HTTP/1.0" 200 OK 2026-03-10 00:39:50,758 - lava_callback - INFO - Received callback for node 69af67a2275c8d3b79399e7c INFO: 172.20.0.1:40010 - "POST /node/69af67a2275c8d3b79399e7c HTTP/1.0" 202 Accepted 2026-03-10 00:39:50,764 - lava_callback - INFO - Starting async job submit for node 69af67a2275c8d3b79399e7c 2026-03-10 00:39:50,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:39:50,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af67a2275c8d3b79399e7c HTTP/1.1" 200 None 2026-03-10 00:39:50,802 - lava_callback - DEBUG - Processing node 69af67a2275c8d3b79399e7c: baseline-arm-clabbe 2026-03-10 00:39:51,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:39:51,174 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:39:51,175 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69af67a2275c8d3b79399e7c/log.txt.gz 2026-03-10 00:39:51,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:39:51,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:39:51,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-10 00:39:51,489 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:39:51,491 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69af67a2275c8d3b79399e7c/lava-logs.yaml 2026-03-10 00:39:51,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:39:51,656 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:39:51,658 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69af67a2275c8d3b79399e7c/lava_callback.json.gz 2026-03-10 00:39:51,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:39:51,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af67a2275c8d3b79399e7c HTTP/1.1" 200 None 2026-03-10 00:39:51,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:39:51,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af632c275c8d3b79399db4 HTTP/1.1" 200 None 2026-03-10 00:39:51,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:39:51,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af67a2275c8d3b79399e7c HTTP/1.1" 200 None 2026-03-10 00:39:51,743 - lava_callback - INFO - Completed processing callback for node 69af67a2275c8d3b79399e7c INFO: 172.20.0.1:40020 - "GET / HTTP/1.0" 200 OK 2026-03-10 00:39:58,496 - lava_callback - INFO - Received callback for node 69acc54bbcd207e915d76d0b INFO: 172.20.0.1:40024 - "POST /node/69acc54bbcd207e915d76d0b HTTP/1.0" 202 Accepted 2026-03-10 00:39:58,502 - lava_callback - INFO - Starting async job submit for node 69acc54bbcd207e915d76d0b 2026-03-10 00:39:58,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:39:58,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acc54bbcd207e915d76d0b HTTP/1.1" 200 None 2026-03-10 00:39:58,519 - lava_callback - DEBUG - Processing node 69acc54bbcd207e915d76d0b: baseline-arm-clabbe 2026-03-10 00:39:58,519 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acc54bbcd207e915d76d0b 2026-03-10 00:39:58,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:39:58,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acc54bbcd207e915d76d0b HTTP/1.1" 200 None 2026-03-10 00:39:58,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:39:58,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa1d368c4d487a0e751145 HTTP/1.1" 200 None 2026-03-10 00:39:58,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:39:58,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acc54bbcd207e915d76d0b HTTP/1.1" 200 None 2026-03-10 00:39:58,577 - lava_callback - ERROR - Error processing callback for node 69acc54bbcd207e915d76d0b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44886 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44898 - "GET / HTTP/1.0" 200 OK 2026-03-10 00:40:21,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:40:21,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:39734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40496 - "GET / HTTP/1.0" 200 OK 2026-03-10 00:41:06,518 - lava_callback - INFO - Received callback for node 69af679d275c8d3b79399e4f INFO: 172.20.0.1:40498 - "POST /node/69af679d275c8d3b79399e4f HTTP/1.0" 202 Accepted 2026-03-10 00:41:06,526 - lava_callback - INFO - Starting async job submit for node 69af679d275c8d3b79399e4f INFO: 172.20.0.1:40514 - "GET / HTTP/1.0" 200 OK 2026-03-10 00:41:06,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:41:06,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af679d275c8d3b79399e4f HTTP/1.1" 200 None 2026-03-10 00:41:06,567 - lava_callback - DEBUG - Processing node 69af679d275c8d3b79399e4f: baseline-arm-broonie 2026-03-10 00:41:06,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:41:06,940 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:41:06,942 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69af679d275c8d3b79399e4f/log.txt.gz 2026-03-10 00:41:07,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:41:07,304 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:41:07,306 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69af679d275c8d3b79399e4f/lava-logs.yaml 2026-03-10 00:41:07,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:41:07,472 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:41:07,474 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69af679d275c8d3b79399e4f/lava_callback.json.gz 2026-03-10 00:41:07,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:41:07,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af679d275c8d3b79399e4f HTTP/1.1" 200 None 2026-03-10 00:41:07,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:41:07,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af632c275c8d3b79399db4 HTTP/1.1" 200 None 2026-03-10 00:41:07,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:41:07,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af679d275c8d3b79399e4f HTTP/1.1" 200 None 2026-03-10 00:41:07,565 - lava_callback - INFO - Completed processing callback for node 69af679d275c8d3b79399e4f INFO: 172.20.0.1:44958 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44974 - "GET / HTTP/1.0" 200 OK 2026-03-10 00:41:21,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:41:21,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:44530 - "GET / HTTP/1.0" 200 OK 2026-03-10 00:41:29,916 - lava_callback - INFO - Received callback for node 69af67a6275c8d3b79399e89 INFO: 172.20.0.1:50630 - "POST /node/69af67a6275c8d3b79399e89 HTTP/1.0" 202 Accepted 2026-03-10 00:41:29,922 - lava_callback - INFO - Starting async job submit for node 69af67a6275c8d3b79399e89 2026-03-10 00:41:29,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:41:29,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af67a6275c8d3b79399e89 HTTP/1.1" 200 None 2026-03-10 00:41:29,958 - lava_callback - DEBUG - Processing node 69af67a6275c8d3b79399e89: baseline-arm-clabbe 2026-03-10 00:41:30,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:41:30,376 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:41:30,378 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69af67a6275c8d3b79399e89/log.txt.gz 2026-03-10 00:41:30,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:41:30,673 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:41:30,675 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69af67a6275c8d3b79399e89/lava-logs.yaml 2026-03-10 00:41:30,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:41:30,840 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:41:30,842 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69af67a6275c8d3b79399e89/lava_callback.json.gz 2026-03-10 00:41:30,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:41:30,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af67a6275c8d3b79399e89 HTTP/1.1" 200 None 2026-03-10 00:41:30,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:41:30,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af632c275c8d3b79399db4 HTTP/1.1" 200 None 2026-03-10 00:41:30,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:41:30,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af67a6275c8d3b79399e89 HTTP/1.1" 200 None 2026-03-10 00:41:30,925 - lava_callback - INFO - Completed processing callback for node 69af67a6275c8d3b79399e89 INFO: 172.20.0.1:50634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59534 - "GET / HTTP/1.0" 200 OK 2026-03-10 00:41:51,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:41:51,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:56090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42970 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36246 - "GET / HTTP/1.0" 200 OK 2026-03-10 00:43:01,012 - lava_callback - INFO - Received callback for node 69af67a7275c8d3b79399e8c INFO: 172.20.0.1:34852 - "POST /node/69af67a7275c8d3b79399e8c HTTP/1.0" 202 Accepted 2026-03-10 00:43:01,018 - lava_callback - INFO - Starting async job submit for node 69af67a7275c8d3b79399e8c 2026-03-10 00:43:01,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:43:01,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af67a7275c8d3b79399e8c HTTP/1.1" 200 None 2026-03-10 00:43:01,054 - lava_callback - DEBUG - Processing node 69af67a7275c8d3b79399e8c: baseline-arm-clabbe 2026-03-10 00:43:01,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:43:01,419 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:43:01,421 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69af67a7275c8d3b79399e8c/log.txt.gz 2026-03-10 00:43:01,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:43:01,757 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:43:01,759 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69af67a7275c8d3b79399e8c/lava-logs.yaml 2026-03-10 00:43:01,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:43:01,927 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:43:01,928 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69af67a7275c8d3b79399e8c/lava_callback.json.gz 2026-03-10 00:43:01,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:43:01,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af67a7275c8d3b79399e8c HTTP/1.1" 200 None 2026-03-10 00:43:01,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:43:01,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af632c275c8d3b79399db4 HTTP/1.1" 200 None 2026-03-10 00:43:01,985 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:43:02,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af67a7275c8d3b79399e8c HTTP/1.1" 200 None 2026-03-10 00:43:02,012 - lava_callback - INFO - Completed processing callback for node 69af67a7275c8d3b79399e8c INFO: 172.20.0.1:34856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54858 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54874 - "GET / HTTP/1.0" 200 OK 2026-03-10 00:43:21,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:43:21,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:53668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52730 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52744 - "GET / HTTP/1.0" 200 OK 2026-03-10 00:44:26,506 - lava_callback - INFO - Received callback for node 69af67d7275c8d3b79399f01 INFO: 172.20.0.1:37570 - "POST /node/69af67d7275c8d3b79399f01 HTTP/1.0" 202 Accepted 2026-03-10 00:44:26,523 - lava_callback - INFO - Starting async job submit for node 69af67d7275c8d3b79399f01 INFO: 172.20.0.1:37572 - "GET / HTTP/1.0" 200 OK 2026-03-10 00:44:26,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:44:26,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af67d7275c8d3b79399f01 HTTP/1.1" 200 None 2026-03-10 00:44:26,619 - lava_callback - DEBUG - Processing node 69af67d7275c8d3b79399f01: ltp-syscalls 2026-03-10 00:44:27,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:44:28,410 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:44:28,413 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-69af67d7275c8d3b79399f01/log.txt.gz 2026-03-10 00:44:29,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:44:29,797 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:44:29,800 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-69af67d7275c8d3b79399f01/lava-logs.yaml 2026-03-10 00:44:29,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:44:30,321 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:44:30,323 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-69af67d7275c8d3b79399f01/lava_callback.json.gz 2026-03-10 00:44:30,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:44:30,389 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af67d7275c8d3b79399f01 HTTP/1.1" 200 None 2026-03-10 00:44:30,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:44:30,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af632c275c8d3b79399db4 HTTP/1.1" 200 None 2026-03-10 00:44:30,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:44:30,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af67d7275c8d3b79399f01 HTTP/1.1" 200 None 2026-03-10 00:44:30,462 - lava_callback - INFO - Completed processing callback for node 69af67d7275c8d3b79399f01 INFO: 172.20.0.1:53088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47912 - "GET / HTTP/1.0" 200 OK 2026-03-10 00:44:51,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:44:51,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:45630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51328 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35054 - "GET / HTTP/1.0" 200 OK 2026-03-10 00:45:30,382 - lava_callback - INFO - Received callback for node 69acc6a0bcd207e915d76d85 INFO: 172.20.0.1:40034 - "POST /node/69acc6a0bcd207e915d76d85 HTTP/1.0" 202 Accepted 2026-03-10 00:45:30,388 - lava_callback - INFO - Starting async job submit for node 69acc6a0bcd207e915d76d85 2026-03-10 00:45:30,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:45:30,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acc6a0bcd207e915d76d85 HTTP/1.1" 200 None 2026-03-10 00:45:30,409 - lava_callback - DEBUG - Processing node 69acc6a0bcd207e915d76d85: baseline-arm-baylibre 2026-03-10 00:45:30,409 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acc6a0bcd207e915d76d85 2026-03-10 00:45:30,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:45:30,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acc6a0bcd207e915d76d85 HTTP/1.1" 200 None 2026-03-10 00:45:30,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:45:30,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a77a57a255cc0324eb010a HTTP/1.1" 200 None 2026-03-10 00:45:30,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:45:30,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acc6a0bcd207e915d76d85 HTTP/1.1" 200 None 2026-03-10 00:45:30,481 - lava_callback - ERROR - Error processing callback for node 69acc6a0bcd207e915d76d85: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40050 - "GET / HTTP/1.0" 200 OK 2026-03-10 00:45:43,657 - lava_callback - INFO - Received callback for node 69af69a5275c8d3b7939a3ae INFO: 172.20.0.1:37282 - "POST /node/69af69a5275c8d3b7939a3ae HTTP/1.0" 202 Accepted 2026-03-10 00:45:43,663 - lava_callback - INFO - Starting async job submit for node 69af69a5275c8d3b7939a3ae 2026-03-10 00:45:43,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:45:43,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af69a5275c8d3b7939a3ae HTTP/1.1" 200 None 2026-03-10 00:45:43,681 - lava_callback - DEBUG - Processing node 69af69a5275c8d3b7939a3ae: baseline-arm 2026-03-10 00:45:43,686 - lava_callback - WARNING - Failed to upload log 2026-03-10 00:45:43,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:45:43,863 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:45:43,865 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69af69a5275c8d3b7939a3ae/lava-logs.yaml 2026-03-10 00:45:43,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:45:43,982 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:45:43,984 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69af69a5275c8d3b7939a3ae/lava_callback.json.gz 2026-03-10 00:45:43,987 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:45:44,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af69a5275c8d3b7939a3ae HTTP/1.1" 200 None 2026-03-10 00:45:44,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:45:44,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af632c275c8d3b79399db4 HTTP/1.1" 200 None 2026-03-10 00:45:44,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:45:44,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af69a5275c8d3b7939a3ae HTTP/1.1" 200 None 2026-03-10 00:45:44,058 - lava_callback - INFO - Completed processing callback for node 69af69a5275c8d3b7939a3ae INFO: 172.20.0.1:37286 - "GET / HTTP/1.0" 200 OK 2026-03-10 00:45:51,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:45:51,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:60000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48620 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36880 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36848 - "GET / HTTP/1.0" 200 OK 2026-03-10 00:47:30,437 - lava_callback - INFO - Received callback for node 69acc70fbcd207e915d76dc0 INFO: 172.20.0.1:42630 - "POST /node/69acc70fbcd207e915d76dc0 HTTP/1.0" 202 Accepted 2026-03-10 00:47:30,443 - lava_callback - INFO - Starting async job submit for node 69acc70fbcd207e915d76dc0 2026-03-10 00:47:30,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:47:30,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acc70fbcd207e915d76dc0 HTTP/1.1" 200 None 2026-03-10 00:47:30,459 - lava_callback - DEBUG - Processing node 69acc70fbcd207e915d76dc0: baseline-arm-baylibre 2026-03-10 00:47:30,460 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acc70fbcd207e915d76dc0 2026-03-10 00:47:30,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:47:30,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acc70fbcd207e915d76dc0 HTTP/1.1" 200 None 2026-03-10 00:47:30,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:47:30,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a77a57a255cc0324eb010a HTTP/1.1" 200 None 2026-03-10 00:47:30,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:47:30,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acc70fbcd207e915d76dc0 HTTP/1.1" 200 None 2026-03-10 00:47:30,513 - lava_callback - ERROR - Error processing callback for node 69acc70fbcd207e915d76dc0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35462 - "GET / HTTP/1.0" 200 OK 2026-03-10 00:48:10,448 - lava_callback - INFO - Received callback for node 69acc735bcd207e915d76df1 INFO: 172.20.0.1:33056 - "POST /node/69acc735bcd207e915d76df1 HTTP/1.0" 202 Accepted 2026-03-10 00:48:10,453 - lava_callback - INFO - Starting async job submit for node 69acc735bcd207e915d76df1 2026-03-10 00:48:10,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:48:10,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acc735bcd207e915d76df1 HTTP/1.1" 200 None 2026-03-10 00:48:10,472 - lava_callback - DEBUG - Processing node 69acc735bcd207e915d76df1: baseline-arm-baylibre 2026-03-10 00:48:10,472 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acc735bcd207e915d76df1 2026-03-10 00:48:10,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:48:10,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acc735bcd207e915d76df1 HTTP/1.1" 200 None 2026-03-10 00:48:10,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:48:10,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a77a57a255cc0324eb010a HTTP/1.1" 200 None 2026-03-10 00:48:10,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:48:10,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acc735bcd207e915d76df1 HTTP/1.1" 200 None 2026-03-10 00:48:10,554 - lava_callback - ERROR - Error processing callback for node 69acc735bcd207e915d76df1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33064 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57698 - "GET / HTTP/1.0" 200 OK 2026-03-10 00:48:30,468 - lava_callback - INFO - Received callback for node 69acc75bbcd207e915d76e11 INFO: 172.20.0.1:43912 - "POST /node/69acc75bbcd207e915d76e11 HTTP/1.0" 202 Accepted 2026-03-10 00:48:30,473 - lava_callback - INFO - Starting async job submit for node 69acc75bbcd207e915d76e11 2026-03-10 00:48:30,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:48:30,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acc75bbcd207e915d76e11 HTTP/1.1" 200 None 2026-03-10 00:48:30,489 - lava_callback - DEBUG - Processing node 69acc75bbcd207e915d76e11: baseline-arm-baylibre 2026-03-10 00:48:30,489 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acc75bbcd207e915d76e11 2026-03-10 00:48:30,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:48:30,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acc75bbcd207e915d76e11 HTTP/1.1" 200 None 2026-03-10 00:48:30,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:48:30,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a77a57a255cc0324eb010a HTTP/1.1" 200 None 2026-03-10 00:48:30,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:48:30,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acc75bbcd207e915d76e11 HTTP/1.1" 200 None 2026-03-10 00:48:30,549 - lava_callback - ERROR - Error processing callback for node 69acc75bbcd207e915d76e11: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40770 - "GET / HTTP/1.0" 200 OK 2026-03-10 00:49:10,517 - lava_callback - INFO - Received callback for node 69acc781bcd207e915d76e21 INFO: 172.20.0.1:32838 - "POST /node/69acc781bcd207e915d76e21 HTTP/1.0" 202 Accepted 2026-03-10 00:49:10,522 - lava_callback - INFO - Starting async job submit for node 69acc781bcd207e915d76e21 2026-03-10 00:49:10,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:49:10,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acc781bcd207e915d76e21 HTTP/1.1" 200 None 2026-03-10 00:49:10,557 - lava_callback - DEBUG - Processing node 69acc781bcd207e915d76e21: baseline-arm-baylibre 2026-03-10 00:49:10,557 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acc781bcd207e915d76e21 2026-03-10 00:49:10,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:49:10,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acc781bcd207e915d76e21 HTTP/1.1" 200 None 2026-03-10 00:49:10,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:49:10,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a77a57a255cc0324eb010a HTTP/1.1" 200 None 2026-03-10 00:49:10,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:49:10,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acc781bcd207e915d76e21 HTTP/1.1" 200 None 2026-03-10 00:49:10,642 - lava_callback - ERROR - Error processing callback for node 69acc781bcd207e915d76e21: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:32852 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32864 - "GET / HTTP/1.0" 200 OK 2026-03-10 00:49:20,235 - lava_callback - INFO - Received callback for node 69af65d2275c8d3b79399dfb INFO: 172.20.0.1:57952 - "POST /node/69af65d2275c8d3b79399dfb HTTP/1.0" 202 Accepted 2026-03-10 00:49:20,241 - lava_callback - INFO - Starting async job submit for node 69af65d2275c8d3b79399dfb 2026-03-10 00:49:20,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:49:20,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af65d2275c8d3b79399dfb HTTP/1.1" 200 None 2026-03-10 00:49:20,360 - lava_callback - DEBUG - Processing node 69af65d2275c8d3b79399dfb: ltp-smoke 2026-03-10 00:49:20,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:49:20,980 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:49:20,982 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69af65d2275c8d3b79399dfb/log.txt.gz 2026-03-10 00:49:21,266 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:49:21,469 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:49:21,471 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69af65d2275c8d3b79399dfb/lava-logs.yaml 2026-03-10 00:49:21,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:49:21,668 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:49:21,669 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69af65d2275c8d3b79399dfb/lava_callback.json.gz 2026-03-10 00:49:21,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:49:21,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af65d2275c8d3b79399dfb HTTP/1.1" 200 None 2026-03-10 00:49:21,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:49:21,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6334275c8d3b79399dba HTTP/1.1" 200 None 2026-03-10 00:49:21,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:49:21,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af65d2275c8d3b79399dfb HTTP/1.1" 200 None 2026-03-10 00:49:22,010 - lava_callback - INFO - Completed processing callback for node 69af65d2275c8d3b79399dfb INFO: 172.20.0.1:57968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32830 - "GET / HTTP/1.0" 200 OK 2026-03-10 00:49:51,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:49:51,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:43864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38158 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44176 - "GET / HTTP/1.0" 200 OK 2026-03-10 00:51:10,658 - lava_callback - INFO - Received callback for node 69acc7f5bcd207e915d76e53 INFO: 172.20.0.1:60276 - "POST /node/69acc7f5bcd207e915d76e53 HTTP/1.0" 202 Accepted 2026-03-10 00:51:10,664 - lava_callback - INFO - Starting async job submit for node 69acc7f5bcd207e915d76e53 2026-03-10 00:51:10,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:51:10,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acc7f5bcd207e915d76e53 HTTP/1.1" 200 None 2026-03-10 00:51:10,689 - lava_callback - DEBUG - Processing node 69acc7f5bcd207e915d76e53: baseline-arm-baylibre 2026-03-10 00:51:10,689 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acc7f5bcd207e915d76e53 2026-03-10 00:51:10,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:51:10,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acc7f5bcd207e915d76e53 HTTP/1.1" 200 None 2026-03-10 00:51:10,704 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:51:10,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a77a57a255cc0324eb010a HTTP/1.1" 200 None 2026-03-10 00:51:10,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:51:10,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acc7f5bcd207e915d76e53 HTTP/1.1" 200 None 2026-03-10 00:51:10,745 - lava_callback - ERROR - Error processing callback for node 69acc7f5bcd207e915d76e53: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60290 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44834 - "GET / HTTP/1.0" 200 OK 2026-03-10 00:51:38,974 - lava_callback - INFO - Received callback for node 69acc813bcd207e915d76eb6 INFO: 172.20.0.1:44848 - "POST /node/69acc813bcd207e915d76eb6 HTTP/1.0" 202 Accepted 2026-03-10 00:51:38,983 - lava_callback - INFO - Starting async job submit for node 69acc813bcd207e915d76eb6 2026-03-10 00:51:38,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:51:39,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acc813bcd207e915d76eb6 HTTP/1.1" 200 None 2026-03-10 00:51:39,018 - lava_callback - DEBUG - Processing node 69acc813bcd207e915d76eb6: baseline-arm-clabbe 2026-03-10 00:51:39,018 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acc813bcd207e915d76eb6 2026-03-10 00:51:39,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:51:39,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acc813bcd207e915d76eb6 HTTP/1.1" 200 None 2026-03-10 00:51:39,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:51:39,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acc009bcd207e915d76b94 HTTP/1.1" 200 None 2026-03-10 00:51:39,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:51:39,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acc813bcd207e915d76eb6 HTTP/1.1" 200 None 2026-03-10 00:51:39,116 - lava_callback - ERROR - Error processing callback for node 69acc813bcd207e915d76eb6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-10 00:51:39,631 - lava_callback - INFO - Received callback for node 69acc814bcd207e915d76eb9 INFO: 172.20.0.1:44862 - "POST /node/69acc814bcd207e915d76eb9 HTTP/1.0" 202 Accepted 2026-03-10 00:51:39,641 - lava_callback - INFO - Starting async job submit for node 69acc814bcd207e915d76eb9 2026-03-10 00:51:39,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:51:39,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acc814bcd207e915d76eb9 HTTP/1.1" 200 None 2026-03-10 00:51:39,682 - lava_callback - DEBUG - Processing node 69acc814bcd207e915d76eb9: baseline-arm-clabbe 2026-03-10 00:51:39,682 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acc814bcd207e915d76eb9 2026-03-10 00:51:39,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:51:39,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acc814bcd207e915d76eb9 HTTP/1.1" 200 None 2026-03-10 00:51:39,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:51:39,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acc009bcd207e915d76b94 HTTP/1.1" 200 None 2026-03-10 00:51:39,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:51:39,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acc814bcd207e915d76eb9 HTTP/1.1" 200 None 2026-03-10 00:51:39,791 - lava_callback - ERROR - Error processing callback for node 69acc814bcd207e915d76eb9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-10 00:51:40,316 - lava_callback - INFO - Received callback for node 69acc819bcd207e915d76ec5 INFO: 172.20.0.1:44504 - "POST /node/69acc819bcd207e915d76ec5 HTTP/1.0" 202 Accepted 2026-03-10 00:51:40,325 - lava_callback - INFO - Starting async job submit for node 69acc819bcd207e915d76ec5 2026-03-10 00:51:40,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:51:40,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acc819bcd207e915d76ec5 HTTP/1.1" 200 None 2026-03-10 00:51:40,355 - lava_callback - DEBUG - Processing node 69acc819bcd207e915d76ec5: baseline-arm-clabbe 2026-03-10 00:51:40,356 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acc819bcd207e915d76ec5 2026-03-10 00:51:40,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:51:40,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acc819bcd207e915d76ec5 HTTP/1.1" 200 None 2026-03-10 00:51:40,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:51:40,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acc009bcd207e915d76b94 HTTP/1.1" 200 None 2026-03-10 00:51:40,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:51:40,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acc819bcd207e915d76ec5 HTTP/1.1" 200 None 2026-03-10 00:51:40,453 - lava_callback - ERROR - Error processing callback for node 69acc819bcd207e915d76ec5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35180 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35790 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42050 - "GET / HTTP/1.0" 200 OK 2026-03-10 00:53:59,515 - lava_callback - INFO - Received callback for node 69af6789275c8d3b79399e1c INFO: 172.20.0.1:42066 - "POST /node/69af6789275c8d3b79399e1c HTTP/1.0" 202 Accepted 2026-03-10 00:53:59,525 - lava_callback - INFO - Starting async job submit for node 69af6789275c8d3b79399e1c 2026-03-10 00:53:59,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:53:59,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6789275c8d3b79399e1c HTTP/1.1" 200 None 2026-03-10 00:53:59,664 - lava_callback - DEBUG - Processing node 69af6789275c8d3b79399e1c: baseline-arm 2026-03-10 00:54:00,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:54:01,278 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:54:01,280 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69af6789275c8d3b79399e1c/log.txt.gz 2026-03-10 00:54:01,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:54:03,089 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:54:03,092 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69af6789275c8d3b79399e1c/lava-logs.yaml 2026-03-10 00:54:03,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:39316 - "GET / HTTP/1.0" 200 OK 2026-03-10 00:54:03,914 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:54:03,916 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69af6789275c8d3b79399e1c/lava_callback.json.gz DEBUG: Setup failed for 69af6789275c8d3b79399e1c. Transitting job node result: incomplete -> fail 2026-03-10 00:54:04,112 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:54:04,149 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6789275c8d3b79399e1c HTTP/1.1" 200 None 2026-03-10 00:54:04,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:54:04,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af632c275c8d3b79399db4 HTTP/1.1" 200 None 2026-03-10 00:54:04,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:54:04,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af6789275c8d3b79399e1c HTTP/1.1" 200 None 2026-03-10 00:54:04,440 - lava_callback - INFO - Completed processing callback for node 69af6789275c8d3b79399e1c INFO: 172.20.0.1:39322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60116 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60130 - "GET / HTTP/1.0" 200 OK 2026-03-10 00:54:21,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:54:21,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:60860 - "GET / HTTP/1.0" 200 OK 2026-03-10 00:54:31,012 - lava_callback - INFO - Received callback for node 69acc8b8bcd207e915d76fe0 INFO: 172.20.0.1:40338 - "POST /node/69acc8b8bcd207e915d76fe0 HTTP/1.0" 202 Accepted 2026-03-10 00:54:31,021 - lava_callback - INFO - Starting async job submit for node 69acc8b8bcd207e915d76fe0 2026-03-10 00:54:31,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:54:31,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acc8b8bcd207e915d76fe0 HTTP/1.1" 200 None 2026-03-10 00:54:31,050 - lava_callback - DEBUG - Processing node 69acc8b8bcd207e915d76fe0: baseline-arm64-baylibre 2026-03-10 00:54:31,051 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acc8b8bcd207e915d76fe0 2026-03-10 00:54:31,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:54:31,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acc8b8bcd207e915d76fe0 HTTP/1.1" 200 None 2026-03-10 00:54:31,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:54:31,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a70fafe31b554803095f11 HTTP/1.1" 200 None 2026-03-10 00:54:31,130 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:54:31,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acc8b8bcd207e915d76fe0 HTTP/1.1" 200 None 2026-03-10 00:54:31,194 - lava_callback - ERROR - Error processing callback for node 69acc8b8bcd207e915d76fe0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39422 - "GET / HTTP/1.0" 200 OK 2026-03-10 00:54:55,032 - lava_callback - INFO - Received callback for node 69af6bc8275c8d3b7939a466 INFO: 172.20.0.1:34474 - "POST /node/69af6bc8275c8d3b7939a466 HTTP/1.0" 202 Accepted 2026-03-10 00:54:55,043 - lava_callback - INFO - Starting async job submit for node 69af6bc8275c8d3b7939a466 2026-03-10 00:54:55,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:54:55,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6bc8275c8d3b7939a466 HTTP/1.1" 200 None 2026-03-10 00:54:55,094 - lava_callback - DEBUG - Processing node 69af6bc8275c8d3b7939a466: baseline-arm64 2026-03-10 00:54:55,103 - lava_callback - WARNING - Failed to upload log 2026-03-10 00:54:55,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:54:55,614 - lava_callback - INFO - Received callback for node 69af6bc9275c8d3b7939a469 INFO: 172.20.0.1:34488 - "POST /node/69af6bc9275c8d3b7939a469 HTTP/1.0" 202 Accepted 2026-03-10 00:54:55,626 - lava_callback - INFO - Starting async job submit for node 69af6bc9275c8d3b7939a469 2026-03-10 00:54:55,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:54:55,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6bc9275c8d3b7939a469 HTTP/1.1" 200 None 2026-03-10 00:54:55,659 - lava_callback - DEBUG - Processing node 69af6bc9275c8d3b7939a469: baseline-arm64 2026-03-10 00:54:55,668 - lava_callback - WARNING - Failed to upload log 2026-03-10 00:54:55,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:54:55,689 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:54:55,690 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af6bc8275c8d3b7939a466/lava-logs.yaml 2026-03-10 00:54:55,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:54:56,104 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:54:56,107 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af6bc9275c8d3b7939a469/lava-logs.yaml 2026-03-10 00:54:56,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:54:56,213 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:54:56,215 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af6bc8275c8d3b7939a466/lava_callback.json.gz 2026-03-10 00:54:56,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:54:56,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6bc8275c8d3b7939a466 HTTP/1.1" 200 None 2026-03-10 00:54:56,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:54:56,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 00:54:56,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:54:56,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af6bc8275c8d3b7939a466 HTTP/1.1" 200 None 2026-03-10 00:54:56,335 - lava_callback - INFO - Completed processing callback for node 69af6bc8275c8d3b7939a466 INFO: 172.20.0.1:34492 - "GET / HTTP/1.0" 200 OK 2026-03-10 00:54:56,704 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:54:56,706 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af6bc9275c8d3b7939a469/lava_callback.json.gz 2026-03-10 00:54:56,711 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:54:56,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6bc9275c8d3b7939a469 HTTP/1.1" 200 None 2026-03-10 00:54:56,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:54:56,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 00:54:56,779 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:54:56,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af6bc9275c8d3b7939a469 HTTP/1.1" 200 None 2026-03-10 00:54:56,844 - lava_callback - INFO - Completed processing callback for node 69af6bc9275c8d3b7939a469 INFO: 172.20.0.1:41280 - "GET / HTTP/1.0" 200 OK 2026-03-10 00:55:04,812 - lava_callback - INFO - Received callback for node 69af6bcb275c8d3b7939a46f INFO: 172.20.0.1:41296 - "POST /node/69af6bcb275c8d3b7939a46f HTTP/1.0" 202 Accepted 2026-03-10 00:55:04,827 - lava_callback - INFO - Starting async job submit for node 69af6bcb275c8d3b7939a46f 2026-03-10 00:55:04,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:55:04,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6bcb275c8d3b7939a46f HTTP/1.1" 200 None 2026-03-10 00:55:04,896 - lava_callback - DEBUG - Processing node 69af6bcb275c8d3b7939a46f: baseline-arm64 2026-03-10 00:55:05,064 - lava_callback - WARNING - Failed to upload log 2026-03-10 00:55:05,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:55:05,576 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:55:05,578 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af6bcb275c8d3b7939a46f/lava-logs.yaml 2026-03-10 00:55:05,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:55:06,142 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:55:06,144 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af6bcb275c8d3b7939a46f/lava_callback.json.gz 2026-03-10 00:55:06,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:55:06,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6bcb275c8d3b7939a46f HTTP/1.1" 200 None 2026-03-10 00:55:06,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:55:06,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 00:55:06,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:55:06,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af6bcb275c8d3b7939a46f HTTP/1.1" 200 None 2026-03-10 00:55:06,383 - lava_callback - INFO - Completed processing callback for node 69af6bcb275c8d3b7939a46f INFO: 172.20.0.1:41312 - "GET / HTTP/1.0" 200 OK 2026-03-10 00:55:12,085 - lava_callback - INFO - Received callback for node 69af6bd6275c8d3b7939a48a INFO: 172.20.0.1:57866 - "POST /node/69af6bd6275c8d3b7939a48a HTTP/1.0" 202 Accepted 2026-03-10 00:55:12,095 - lava_callback - INFO - Starting async job submit for node 69af6bd6275c8d3b7939a48a 2026-03-10 00:55:12,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:55:12,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6bd6275c8d3b7939a48a HTTP/1.1" 200 None 2026-03-10 00:55:12,136 - lava_callback - DEBUG - Processing node 69af6bd6275c8d3b7939a48a: baseline-arm64 2026-03-10 00:55:12,147 - lava_callback - WARNING - Failed to upload log 2026-03-10 00:55:12,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:55:12,678 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:55:12,680 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af6bd6275c8d3b7939a48a/lava-logs.yaml 2026-03-10 00:55:12,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:55:13,286 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:55:13,288 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af6bd6275c8d3b7939a48a/lava_callback.json.gz 2026-03-10 00:55:13,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:55:13,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6bd6275c8d3b7939a48a HTTP/1.1" 200 None 2026-03-10 00:55:13,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:55:13,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 00:55:13,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:55:13,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af6bd6275c8d3b7939a48a HTTP/1.1" 200 None 2026-03-10 00:55:13,397 - lava_callback - INFO - Completed processing callback for node 69af6bd6275c8d3b7939a48a INFO: 172.20.0.1:57868 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57874 - "GET / HTTP/1.0" 200 OK 2026-03-10 00:55:21,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:55:21,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-10 00:55:24,961 - lava_callback - INFO - Received callback for node 69af6bc7275c8d3b7939a463 INFO: 172.20.0.1:53550 - "POST /node/69af6bc7275c8d3b7939a463 HTTP/1.0" 202 Accepted 2026-03-10 00:55:24,970 - lava_callback - INFO - Starting async job submit for node 69af6bc7275c8d3b7939a463 2026-03-10 00:55:25,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:55:25,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6bc7275c8d3b7939a463 HTTP/1.1" 200 None 2026-03-10 00:55:25,043 - lava_callback - DEBUG - Processing node 69af6bc7275c8d3b7939a463: baseline-arm64 2026-03-10 00:55:25,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:55:25,703 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:55:25,705 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af6bc7275c8d3b7939a463/log.txt.gz 2026-03-10 00:55:25,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:55:26,450 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:55:26,453 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af6bc7275c8d3b7939a463/lava-logs.yaml 2026-03-10 00:55:26,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:53564 - "GET / HTTP/1.0" 200 OK 2026-03-10 00:55:26,742 - lava_callback - INFO - Received callback for node 69af6bcd275c8d3b7939a472 INFO: 172.20.0.1:53576 - "POST /node/69af6bcd275c8d3b7939a472 HTTP/1.0" 202 Accepted 2026-03-10 00:55:26,756 - lava_callback - INFO - Starting async job submit for node 69af6bcd275c8d3b7939a472 2026-03-10 00:55:26,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:55:26,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6bcd275c8d3b7939a472 HTTP/1.1" 200 None 2026-03-10 00:55:26,831 - lava_callback - DEBUG - Processing node 69af6bcd275c8d3b7939a472: baseline-arm64 2026-03-10 00:55:26,915 - lava_callback - WARNING - Failed to upload log 2026-03-10 00:55:26,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:55:27,129 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:55:27,132 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af6bc7275c8d3b7939a463/lava_callback.json.gz 2026-03-10 00:55:27,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:55:27,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6bc7275c8d3b7939a463 HTTP/1.1" 200 None 2026-03-10 00:55:27,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:55:27,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 00:55:27,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:55:27,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af6bc7275c8d3b7939a463 HTTP/1.1" 200 None 2026-03-10 00:55:27,293 - lava_callback - INFO - Completed processing callback for node 69af6bc7275c8d3b7939a463 2026-03-10 00:55:27,474 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:55:27,477 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af6bcd275c8d3b7939a472/lava-logs.yaml 2026-03-10 00:55:27,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:55:27,649 - lava_callback - INFO - Received callback for node 69af6bea275c8d3b7939a4d5 INFO: 172.20.0.1:53590 - "POST /node/69af6bea275c8d3b7939a4d5 HTTP/1.0" 202 Accepted 2026-03-10 00:55:27,659 - lava_callback - INFO - Starting async job submit for node 69af6bea275c8d3b7939a4d5 2026-03-10 00:55:27,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:55:27,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6bea275c8d3b7939a4d5 HTTP/1.1" 200 None 2026-03-10 00:55:27,719 - lava_callback - DEBUG - Processing node 69af6bea275c8d3b7939a4d5: baseline-nfs-arm64 2026-03-10 00:55:27,731 - lava_callback - WARNING - Failed to upload log 2026-03-10 00:55:27,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:55:27,985 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:55:27,987 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af6bcd275c8d3b7939a472/lava_callback.json.gz 2026-03-10 00:55:28,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:55:28,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6bcd275c8d3b7939a472 HTTP/1.1" 200 None 2026-03-10 00:55:28,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:55:28,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 00:55:28,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:55:28,112 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:55:28,114 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af6bea275c8d3b7939a4d5/lava-logs.yaml 2026-03-10 00:55:28,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:55:28,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af6bcd275c8d3b7939a472 HTTP/1.1" 200 None 2026-03-10 00:55:28,196 - lava_callback - INFO - Completed processing callback for node 69af6bcd275c8d3b7939a472 2026-03-10 00:55:28,265 - lava_callback - INFO - Received callback for node 69af6bd5275c8d3b7939a487 INFO: 172.20.0.1:53602 - "POST /node/69af6bd5275c8d3b7939a487 HTTP/1.0" 202 Accepted 2026-03-10 00:55:28,296 - lava_callback - INFO - Starting async job submit for node 69af6bd5275c8d3b7939a487 2026-03-10 00:55:28,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:55:28,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6bd5275c8d3b7939a487 HTTP/1.1" 200 None 2026-03-10 00:55:28,428 - lava_callback - DEBUG - Processing node 69af6bd5275c8d3b7939a487: baseline-arm64 2026-03-10 00:55:28,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:55:28,951 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:55:28,953 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af6bea275c8d3b7939a4d5/lava_callback.json.gz 2026-03-10 00:55:28,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:55:28,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6bea275c8d3b7939a4d5 HTTP/1.1" 200 None 2026-03-10 00:55:28,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:55:29,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 00:55:29,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:55:29,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af6bea275c8d3b7939a4d5 HTTP/1.1" 200 None 2026-03-10 00:55:29,066 - lava_callback - INFO - Completed processing callback for node 69af6bea275c8d3b7939a4d5 2026-03-10 00:55:29,328 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:55:29,330 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af6bd5275c8d3b7939a487/log.txt.gz 2026-03-10 00:55:29,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:55:30,237 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:55:30,239 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af6bd5275c8d3b7939a487/lava-logs.yaml 2026-03-10 00:55:30,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:55:30,740 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:55:30,743 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af6bd5275c8d3b7939a487/lava_callback.json.gz 2026-03-10 00:55:30,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:55:30,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6bd5275c8d3b7939a487 HTTP/1.1" 200 None 2026-03-10 00:55:30,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:55:30,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 00:55:30,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:55:30,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af6bd5275c8d3b7939a487 HTTP/1.1" 200 None 2026-03-10 00:55:30,885 - lava_callback - INFO - Completed processing callback for node 69af6bd5275c8d3b7939a487 2026-03-10 00:55:32,223 - lava_callback - INFO - Received callback for node 69af6be8275c8d3b7939a4d0 INFO: 172.20.0.1:53634 - "POST /node/69af6be8275c8d3b7939a4d0 HTTP/1.0" 202 Accepted 2026-03-10 00:55:32,232 - lava_callback - INFO - Starting async job submit for node 69af6be8275c8d3b7939a4d0 2026-03-10 00:55:32,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:55:32,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6be8275c8d3b7939a4d0 HTTP/1.1" 200 None 2026-03-10 00:55:32,269 - lava_callback - DEBUG - Processing node 69af6be8275c8d3b7939a4d0: baseline-nfs-arm64 2026-03-10 00:55:32,271 - lava_callback - WARNING - Failed to upload log 2026-03-10 00:55:32,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:55:32,500 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:55:32,502 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af6be8275c8d3b7939a4d0/lava-logs.yaml 2026-03-10 00:55:32,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:55:32,631 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:55:32,633 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af6be8275c8d3b7939a4d0/lava_callback.json.gz 2026-03-10 00:55:32,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:55:32,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6be8275c8d3b7939a4d0 HTTP/1.1" 200 None 2026-03-10 00:55:32,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:55:32,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 00:55:32,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:55:32,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af6be8275c8d3b7939a4d0 HTTP/1.1" 200 None 2026-03-10 00:55:32,717 - lava_callback - INFO - Completed processing callback for node 69af6be8275c8d3b7939a4d0 2026-03-10 00:55:34,225 - lava_callback - INFO - Received callback for node 69af6bde275c8d3b7939a4b3 INFO: 172.20.0.1:53650 - "POST /node/69af6bde275c8d3b7939a4b3 HTTP/1.0" 202 Accepted 2026-03-10 00:55:34,235 - lava_callback - INFO - Starting async job submit for node 69af6bde275c8d3b7939a4b3 2026-03-10 00:55:34,339 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:55:34,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6bde275c8d3b7939a4b3 HTTP/1.1" 200 None 2026-03-10 00:55:34,384 - lava_callback - DEBUG - Processing node 69af6bde275c8d3b7939a4b3: baseline-arm64 2026-03-10 00:55:34,443 - lava_callback - WARNING - Failed to upload log 2026-03-10 00:55:34,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:55:34,632 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:55:34,635 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af6bde275c8d3b7939a4b3/lava-logs.yaml 2026-03-10 00:55:34,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:55:34,777 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:55:34,779 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af6bde275c8d3b7939a4b3/lava_callback.json.gz 2026-03-10 00:55:34,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:55:34,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6bde275c8d3b7939a4b3 HTTP/1.1" 200 None 2026-03-10 00:55:34,830 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:55:34,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 00:55:34,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:55:34,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af6bde275c8d3b7939a4b3 HTTP/1.1" 200 None 2026-03-10 00:55:34,925 - lava_callback - INFO - Completed processing callback for node 69af6bde275c8d3b7939a4b3 INFO: 172.20.0.1:53652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57128 - "GET / HTTP/1.0" 200 OK 2026-03-10 00:55:51,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:55:51,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.20.0.1:39756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43210 - "GET / HTTP/1.0" 200 OK 2026-03-10 00:56:06,943 - lava_callback - INFO - Received callback for node 69af6c0b275c8d3b7939a548 INFO: 172.20.0.1:43222 - "POST /node/69af6c0b275c8d3b7939a548 HTTP/1.0" 202 Accepted 2026-03-10 00:56:06,951 - lava_callback - INFO - Starting async job submit for node 69af6c0b275c8d3b7939a548 2026-03-10 00:56:06,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:56:06,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6c0b275c8d3b7939a548 HTTP/1.1" 200 None 2026-03-10 00:56:06,987 - lava_callback - DEBUG - Processing node 69af6c0b275c8d3b7939a548: baseline-arm64-broonie 2026-03-10 00:56:06,995 - lava_callback - WARNING - Failed to upload log 2026-03-10 00:56:07,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:56:07,246 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:56:07,248 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af6c0b275c8d3b7939a548/lava-logs.yaml 2026-03-10 00:56:07,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:56:07,472 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:56:07,474 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af6c0b275c8d3b7939a548/lava_callback.json.gz 2026-03-10 00:56:07,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:56:07,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6c0b275c8d3b7939a548 HTTP/1.1" 200 None 2026-03-10 00:56:07,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:56:07,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 00:56:07,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:56:07,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af6c0b275c8d3b7939a548 HTTP/1.1" 200 None 2026-03-10 00:56:07,577 - lava_callback - INFO - Completed processing callback for node 69af6c0b275c8d3b7939a548 2026-03-10 00:56:09,135 - lava_callback - INFO - Received callback for node 69af6c12275c8d3b7939a55b INFO: 172.20.0.1:43236 - "POST /node/69af6c12275c8d3b7939a55b HTTP/1.0" 202 Accepted 2026-03-10 00:56:09,144 - lava_callback - INFO - Starting async job submit for node 69af6c12275c8d3b7939a55b 2026-03-10 00:56:09,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:56:09,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6c12275c8d3b7939a55b HTTP/1.1" 200 None 2026-03-10 00:56:09,174 - lava_callback - DEBUG - Processing node 69af6c12275c8d3b7939a55b: baseline-arm64-clabbe 2026-03-10 00:56:09,181 - lava_callback - WARNING - Failed to upload log 2026-03-10 00:56:09,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:56:09,428 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:56:09,430 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69af6c12275c8d3b7939a55b/lava-logs.yaml 2026-03-10 00:56:09,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:56:09,552 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:56:09,553 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69af6c12275c8d3b7939a55b/lava_callback.json.gz 2026-03-10 00:56:09,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:56:09,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6c12275c8d3b7939a55b HTTP/1.1" 200 None 2026-03-10 00:56:09,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:56:09,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 00:56:09,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:56:09,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af6c12275c8d3b7939a55b HTTP/1.1" 200 None 2026-03-10 00:56:09,637 - lava_callback - INFO - Completed processing callback for node 69af6c12275c8d3b7939a55b INFO: 172.20.0.1:58100 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58104 - "GET / HTTP/1.0" 200 OK 2026-03-10 00:56:21,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:56:21,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-10 00:56:24,465 - lava_callback - INFO - Received callback for node 69af6bd9275c8d3b7939a4aa INFO: 172.20.0.1:57336 - "POST /node/69af6bd9275c8d3b7939a4aa HTTP/1.0" 202 Accepted 2026-03-10 00:56:24,481 - lava_callback - INFO - Starting async job submit for node 69af6bd9275c8d3b7939a4aa 2026-03-10 00:56:24,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:56:24,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6bd9275c8d3b7939a4aa HTTP/1.1" 200 None 2026-03-10 00:56:24,581 - lava_callback - DEBUG - Processing node 69af6bd9275c8d3b7939a4aa: baseline-arm64 2026-03-10 00:56:25,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:56:25,399 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:56:25,401 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af6bd9275c8d3b7939a4aa/log.txt.gz 2026-03-10 00:56:25,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:56:26,138 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:56:26,140 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af6bd9275c8d3b7939a4aa/lava-logs.yaml 2026-03-10 00:56:26,147 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:56:26,273 - lava_callback - INFO - Received callback for node 69af6c0d275c8d3b7939a54b INFO: 172.20.0.1:57348 - "POST /node/69af6c0d275c8d3b7939a54b HTTP/1.0" 202 Accepted 2026-03-10 00:56:26,279 - lava_callback - INFO - Starting async job submit for node 69af6c0d275c8d3b7939a54b 2026-03-10 00:56:26,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:56:26,295 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:56:26,297 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af6bd9275c8d3b7939a4aa/lava_callback.json.gz 2026-03-10 00:56:26,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6c0d275c8d3b7939a54b HTTP/1.1" 200 None 2026-03-10 00:56:26,341 - lava_callback - DEBUG - Processing node 69af6c0d275c8d3b7939a54b: baseline-arm64-broonie DEBUG: Setup failed for 69af6bd9275c8d3b7939a4aa. Transitting job node result: incomplete -> fail 2026-03-10 00:56:26,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:56:26,382 - lava_callback - WARNING - Failed to upload log 2026-03-10 00:56:26,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:56:26,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6bd9275c8d3b7939a4aa HTTP/1.1" 200 None 2026-03-10 00:56:26,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:56:26,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 00:56:26,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:56:26,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af6bd9275c8d3b7939a4aa HTTP/1.1" 200 None INFO: 172.20.0.1:57364 - "GET / HTTP/1.0" 200 OK 2026-03-10 00:56:26,629 - lava_callback - INFO - Completed processing callback for node 69af6bd9275c8d3b7939a4aa 2026-03-10 00:56:27,090 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:56:27,092 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af6c0d275c8d3b7939a54b/lava-logs.yaml 2026-03-10 00:56:27,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:56:27,270 - lava_callback - INFO - Received callback for node 69af6bdb275c8d3b7939a4ad INFO: 172.20.0.1:57374 - "POST /node/69af6bdb275c8d3b7939a4ad HTTP/1.0" 202 Accepted 2026-03-10 00:56:27,281 - lava_callback - INFO - Starting async job submit for node 69af6bdb275c8d3b7939a4ad 2026-03-10 00:56:27,334 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:56:27,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6bdb275c8d3b7939a4ad HTTP/1.1" 200 None 2026-03-10 00:56:27,351 - lava_callback - DEBUG - Processing node 69af6bdb275c8d3b7939a4ad: baseline-arm64 2026-03-10 00:56:27,432 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:56:27,434 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af6c0d275c8d3b7939a54b/lava_callback.json.gz 2026-03-10 00:56:27,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:56:27,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6c0d275c8d3b7939a54b HTTP/1.1" 200 None 2026-03-10 00:56:27,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:56:27,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 00:56:27,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:56:27,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af6c0d275c8d3b7939a54b HTTP/1.1" 200 None 2026-03-10 00:56:27,865 - lava_callback - INFO - Received callback for node 69af6bdc275c8d3b7939a4b0 2026-03-10 00:56:27,873 - lava_callback - INFO - Completed processing callback for node 69af6c0d275c8d3b7939a54b INFO: 172.20.0.1:57384 - "POST /node/69af6bdc275c8d3b7939a4b0 HTTP/1.0" 202 Accepted 2026-03-10 00:56:27,903 - lava_callback - INFO - Starting async job submit for node 69af6bdc275c8d3b7939a4b0 2026-03-10 00:56:28,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:56:28,164 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6bdc275c8d3b7939a4b0 HTTP/1.1" 200 None 2026-03-10 00:56:28,167 - lava_callback - DEBUG - Processing node 69af6bdc275c8d3b7939a4b0: baseline-arm64 2026-03-10 00:56:28,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:56:28,682 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:56:28,723 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af6bdb275c8d3b7939a4ad/log.txt.gz 2026-03-10 00:56:28,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:56:29,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:56:29,323 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:56:29,325 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af6bdc275c8d3b7939a4b0/log.txt.gz 2026-03-10 00:56:29,601 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:56:29,619 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af6bdb275c8d3b7939a4ad/lava-logs.yaml 2026-03-10 00:56:29,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:56:29,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:56:30,031 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:56:30,034 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af6bdc275c8d3b7939a4b0/lava-logs.yaml 2026-03-10 00:56:30,036 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:56:30,038 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af6bdb275c8d3b7939a4ad/lava_callback.json.gz 2026-03-10 00:56:30,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 DEBUG: Setup failed for 69af6bdb275c8d3b7939a4ad. Transitting job node result: incomplete -> fail 2026-03-10 00:56:30,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:56:30,166 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6bdb275c8d3b7939a4ad HTTP/1.1" 200 None 2026-03-10 00:56:30,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:56:30,193 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 00:56:30,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:56:30,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af6bdb275c8d3b7939a4ad HTTP/1.1" 200 None 2026-03-10 00:56:30,336 - lava_callback - INFO - Completed processing callback for node 69af6bdb275c8d3b7939a4ad 2026-03-10 00:56:30,660 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:56:30,662 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af6bdc275c8d3b7939a4b0/lava_callback.json.gz DEBUG: Setup failed for 69af6bdc275c8d3b7939a4b0. Transitting job node result: incomplete -> fail 2026-03-10 00:56:30,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:56:30,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6bdc275c8d3b7939a4b0 HTTP/1.1" 200 None 2026-03-10 00:56:30,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:56:30,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 00:56:30,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:56:30,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af6bdc275c8d3b7939a4b0 HTTP/1.1" 200 None 2026-03-10 00:56:30,934 - lava_callback - INFO - Completed processing callback for node 69af6bdc275c8d3b7939a4b0 INFO: 172.20.0.1:60018 - "GET / HTTP/1.0" 200 OK 2026-03-10 00:56:39,161 - lava_callback - INFO - Received callback for node 69af6bd2275c8d3b7939a481 INFO: 172.20.0.1:60022 - "POST /node/69af6bd2275c8d3b7939a481 HTTP/1.0" 202 Accepted 2026-03-10 00:56:39,171 - lava_callback - INFO - Starting async job submit for node 69af6bd2275c8d3b7939a481 2026-03-10 00:56:39,240 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:56:39,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6bd2275c8d3b7939a481 HTTP/1.1" 200 None 2026-03-10 00:56:39,266 - lava_callback - DEBUG - Processing node 69af6bd2275c8d3b7939a481: baseline-arm64 2026-03-10 00:56:39,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:56:40,117 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:56:40,120 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af6bd2275c8d3b7939a481/log.txt.gz 2026-03-10 00:56:40,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:56:40,921 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:56:40,923 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af6bd2275c8d3b7939a481/lava-logs.yaml 2026-03-10 00:56:40,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:56:41,063 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:56:41,065 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af6bd2275c8d3b7939a481/lava_callback.json.gz DEBUG: Setup failed for 69af6bd2275c8d3b7939a481. Transitting job node result: incomplete -> fail 2026-03-10 00:56:41,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:56:41,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6bd2275c8d3b7939a481 HTTP/1.1" 200 None 2026-03-10 00:56:41,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:56:41,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 00:56:41,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:56:41,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af6bd2275c8d3b7939a481 HTTP/1.1" 200 None 2026-03-10 00:56:41,335 - lava_callback - INFO - Completed processing callback for node 69af6bd2275c8d3b7939a481 2026-03-10 00:56:42,315 - lava_callback - INFO - Received callback for node 69af6bef275c8d3b7939a4e2 INFO: 172.20.0.1:38816 - "POST /node/69af6bef275c8d3b7939a4e2 HTTP/1.0" 202 Accepted 2026-03-10 00:56:42,326 - lava_callback - INFO - Starting async job submit for node 69af6bef275c8d3b7939a4e2 2026-03-10 00:56:42,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:56:42,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6bef275c8d3b7939a4e2 HTTP/1.1" 200 None 2026-03-10 00:56:42,421 - lava_callback - DEBUG - Processing node 69af6bef275c8d3b7939a4e2: baseline-nfs-arm64 2026-03-10 00:56:42,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:56:43,291 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:56:43,293 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af6bef275c8d3b7939a4e2/log.txt.gz 2026-03-10 00:56:43,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:56:44,064 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:56:44,067 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af6bef275c8d3b7939a4e2/lava-logs.yaml 2026-03-10 00:56:44,076 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:56:44,196 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:56:44,197 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af6bef275c8d3b7939a4e2/lava_callback.json.gz DEBUG: Setup failed for 69af6bef275c8d3b7939a4e2. Transitting job node result: incomplete -> fail 2026-03-10 00:56:44,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:56:44,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6bef275c8d3b7939a4e2 HTTP/1.1" 200 None 2026-03-10 00:56:44,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:56:44,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 00:56:44,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:56:44,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af6bef275c8d3b7939a4e2 HTTP/1.1" 200 None 2026-03-10 00:56:44,498 - lava_callback - INFO - Completed processing callback for node 69af6bef275c8d3b7939a4e2 INFO: 172.20.0.1:38828 - "GET / HTTP/1.0" 200 OK 2026-03-10 00:56:51,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:56:51,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-10 00:56:54,233 - lava_callback - INFO - Received callback for node 69af6be6275c8d3b7939a4ca INFO: 172.20.0.1:53502 - "POST /node/69af6be6275c8d3b7939a4ca HTTP/1.0" 202 Accepted 2026-03-10 00:56:54,243 - lava_callback - INFO - Starting async job submit for node 69af6be6275c8d3b7939a4ca 2026-03-10 00:56:54,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:56:54,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6be6275c8d3b7939a4ca HTTP/1.1" 200 None 2026-03-10 00:56:54,345 - lava_callback - DEBUG - Processing node 69af6be6275c8d3b7939a4ca: baseline-nfs-arm64 2026-03-10 00:56:55,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:56:55,146 - lava_callback - INFO - Received callback for node 69af6bd1275c8d3b7939a47e 2026-03-10 00:56:55,164 - lava_callback - INFO - Starting async job submit for node 69af6bd1275c8d3b7939a47e INFO: 172.20.0.1:53506 - "POST /node/69af6bd1275c8d3b7939a47e HTTP/1.0" 202 Accepted 2026-03-10 00:56:55,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:56:55,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6bd1275c8d3b7939a47e HTTP/1.1" 200 None 2026-03-10 00:56:55,267 - lava_callback - DEBUG - Processing node 69af6bd1275c8d3b7939a47e: baseline-arm64 2026-03-10 00:56:55,448 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:56:55,511 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af6be6275c8d3b7939a4ca/log.txt.gz 2026-03-10 00:56:55,854 - lava_callback - INFO - Received callback for node 69af6bed275c8d3b7939a4dc 2026-03-10 00:56:55,879 - lava_callback - INFO - Starting async job submit for node 69af6bed275c8d3b7939a4dc INFO: 172.20.0.1:53516 - "POST /node/69af6bed275c8d3b7939a4dc HTTP/1.0" 202 Accepted 2026-03-10 00:56:56,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:56:56,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6bed275c8d3b7939a4dc HTTP/1.1" 200 None 2026-03-10 00:56:56,109 - lava_callback - DEBUG - Processing node 69af6bed275c8d3b7939a4dc: baseline-nfs-arm64 INFO: 172.20.0.1:53526 - "GET / HTTP/1.0" 200 OK 2026-03-10 00:56:57,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:56:57,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:56:57,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:56:57,824 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:56:57,825 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:56:57,828 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af6bed275c8d3b7939a4dc/log.txt.gz 2026-03-10 00:56:57,829 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af6bd1275c8d3b7939a47e/log.txt.gz 2026-03-10 00:56:57,910 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:56:58,001 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af6be6275c8d3b7939a4ca/lava-logs.yaml 2026-03-10 00:56:58,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:56:58,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:56:58,671 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:56:58,710 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af6be6275c8d3b7939a4ca/lava_callback.json.gz 2026-03-10 00:56:58,826 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 DEBUG: Setup failed for 69af6be6275c8d3b7939a4ca. Transitting job node result: incomplete -> fail 2026-03-10 00:56:58,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:56:58,859 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af6bed275c8d3b7939a4dc/lava-logs.yaml 2026-03-10 00:56:58,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6be6275c8d3b7939a4ca HTTP/1.1" 200 None 2026-03-10 00:56:58,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:56:58,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:56:58,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:56:58,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 00:56:58,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:56:59,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af6be6275c8d3b7939a4ca HTTP/1.1" 200 None 2026-03-10 00:56:59,048 - lava_callback - INFO - Received callback for node 69af6c16275c8d3b7939a57b 2026-03-10 00:56:59,091 - lava_callback - INFO - Starting async job submit for node 69af6c16275c8d3b7939a57b INFO: 172.20.0.1:53540 - "POST /node/69af6c16275c8d3b7939a57b HTTP/1.0" 202 Accepted 2026-03-10 00:56:59,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:56:59,235 - lava_callback - INFO - Completed processing callback for node 69af6be6275c8d3b7939a4ca 2026-03-10 00:56:59,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6c16275c8d3b7939a57b HTTP/1.1" 200 None 2026-03-10 00:56:59,269 - lava_callback - DEBUG - Processing node 69af6c16275c8d3b7939a57b: baseline-arm64-pengutronix 2026-03-10 00:56:59,585 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:56:59,604 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af6bed275c8d3b7939a4dc/lava_callback.json.gz 2026-03-10 00:56:59,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:56:59,659 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 DEBUG: Setup failed for 69af6bed275c8d3b7939a4dc. Transitting job node result: incomplete -> fail 2026-03-10 00:56:59,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:56:59,683 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af6bd1275c8d3b7939a47e/lava-logs.yaml 2026-03-10 00:56:59,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:56:59,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6bed275c8d3b7939a4dc HTTP/1.1" 200 None 2026-03-10 00:56:59,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:56:59,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 00:56:59,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:56:59,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af6bed275c8d3b7939a4dc HTTP/1.1" 200 None 2026-03-10 00:56:59,861 - lava_callback - INFO - Completed processing callback for node 69af6bed275c8d3b7939a4dc 2026-03-10 00:57:00,213 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:57:00,215 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69af6c16275c8d3b7939a57b/log.txt.gz 2026-03-10 00:57:00,368 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:57:00,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:57:00,387 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af6bd1275c8d3b7939a47e/lava_callback.json.gz DEBUG: Setup failed for 69af6bd1275c8d3b7939a47e. Transitting job node result: incomplete -> fail 2026-03-10 00:57:00,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:57:00,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6bd1275c8d3b7939a47e HTTP/1.1" 200 None 2026-03-10 00:57:00,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:57:00,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 00:57:00,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:57:00,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af6bd1275c8d3b7939a47e HTTP/1.1" 200 None 2026-03-10 00:57:00,613 - lava_callback - INFO - Completed processing callback for node 69af6bd1275c8d3b7939a47e 2026-03-10 00:57:01,020 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:57:01,022 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69af6c16275c8d3b7939a57b/lava-logs.yaml 2026-03-10 00:57:01,030 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:57:01,205 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:57:01,207 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69af6c16275c8d3b7939a57b/lava_callback.json.gz 2026-03-10 00:57:01,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:57:01,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6c16275c8d3b7939a57b HTTP/1.1" 200 None 2026-03-10 00:57:01,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:57:01,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 00:57:01,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:57:01,287 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af6c16275c8d3b7939a57b HTTP/1.1" 200 None 2026-03-10 00:57:01,298 - lava_callback - INFO - Completed processing callback for node 69af6c16275c8d3b7939a57b INFO: 172.20.0.1:48080 - "GET / HTTP/1.0" 200 OK 2026-03-10 00:57:04,896 - lava_callback - INFO - Received callback for node 69af6c17275c8d3b7939a588 INFO: 172.20.0.1:48084 - "POST /node/69af6c17275c8d3b7939a588 HTTP/1.0" 202 Accepted 2026-03-10 00:57:04,903 - lava_callback - INFO - Starting async job submit for node 69af6c17275c8d3b7939a588 2026-03-10 00:57:04,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:57:04,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6c17275c8d3b7939a588 HTTP/1.1" 200 None 2026-03-10 00:57:04,942 - lava_callback - DEBUG - Processing node 69af6c17275c8d3b7939a588: baseline-arm64-pengutronix 2026-03-10 00:57:05,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:57:05,448 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:57:05,450 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69af6c17275c8d3b7939a588/log.txt.gz 2026-03-10 00:57:05,573 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:57:05,796 - lava_callback - INFO - Received callback for node 69af6bd4275c8d3b7939a484 INFO: 172.20.0.1:48100 - "POST /node/69af6bd4275c8d3b7939a484 HTTP/1.0" 202 Accepted 2026-03-10 00:57:05,806 - lava_callback - INFO - Starting async job submit for node 69af6bd4275c8d3b7939a484 2026-03-10 00:57:05,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:57:05,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6bd4275c8d3b7939a484 HTTP/1.1" 200 None 2026-03-10 00:57:05,895 - lava_callback - DEBUG - Processing node 69af6bd4275c8d3b7939a484: baseline-arm64 2026-03-10 00:57:06,179 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:57:06,222 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69af6c17275c8d3b7939a588/lava-logs.yaml 2026-03-10 00:57:06,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:48114 - "GET / HTTP/1.0" 200 OK 2026-03-10 00:57:06,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:57:06,818 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:57:06,820 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69af6c17275c8d3b7939a588/lava_callback.json.gz 2026-03-10 00:57:06,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:57:06,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6c17275c8d3b7939a588 HTTP/1.1" 200 None 2026-03-10 00:57:06,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:57:06,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 00:57:06,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:57:06,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af6c17275c8d3b7939a588 HTTP/1.1" 200 None 2026-03-10 00:57:06,952 - lava_callback - INFO - Completed processing callback for node 69af6c17275c8d3b7939a588 2026-03-10 00:57:07,060 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:57:07,062 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af6bd4275c8d3b7939a484/log.txt.gz 2026-03-10 00:57:07,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:57:07,880 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:57:07,882 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af6bd4275c8d3b7939a484/lava-logs.yaml 2026-03-10 00:57:07,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:57:08,330 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:57:08,332 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af6bd4275c8d3b7939a484/lava_callback.json.gz DEBUG: Setup failed for 69af6bd4275c8d3b7939a484. Transitting job node result: incomplete -> fail 2026-03-10 00:57:08,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:57:08,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6bd4275c8d3b7939a484 HTTP/1.1" 200 None 2026-03-10 00:57:08,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:57:08,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 00:57:08,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:57:08,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af6bd4275c8d3b7939a484 HTTP/1.1" 200 None 2026-03-10 00:57:08,557 - lava_callback - INFO - Completed processing callback for node 69af6bd4275c8d3b7939a484 2026-03-10 00:57:10,681 - lava_callback - INFO - Received callback for node 69af6be4275c8d3b7939a4c7 INFO: 172.20.0.1:46024 - "POST /node/69af6be4275c8d3b7939a4c7 HTTP/1.0" 202 Accepted 2026-03-10 00:57:10,693 - lava_callback - INFO - Starting async job submit for node 69af6be4275c8d3b7939a4c7 2026-03-10 00:57:10,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:57:10,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6be4275c8d3b7939a4c7 HTTP/1.1" 200 None 2026-03-10 00:57:10,801 - lava_callback - DEBUG - Processing node 69af6be4275c8d3b7939a4c7: baseline-nfs-arm64 2026-03-10 00:57:11,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:57:12,171 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:57:12,173 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af6be4275c8d3b7939a4c7/log.txt.gz 2026-03-10 00:57:12,569 - lava_callback - INFO - Received callback for node 69af6c18275c8d3b7939a58b INFO: 172.20.0.1:46030 - "POST /node/69af6c18275c8d3b7939a58b HTTP/1.0" 202 Accepted 2026-03-10 00:57:12,600 - lava_callback - INFO - Received callback for node 69af6bf0275c8d3b7939a4e6 2026-03-10 00:57:12,612 - lava_callback - INFO - Starting async job submit for node 69af6c18275c8d3b7939a58b INFO: 172.20.0.1:46040 - "POST /node/69af6bf0275c8d3b7939a4e6 HTTP/1.0" 202 Accepted 2026-03-10 00:57:12,640 - lava_callback - INFO - Starting async job submit for node 69af6bf0275c8d3b7939a4e6 2026-03-10 00:57:12,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:57:12,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:57:12,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6c18275c8d3b7939a58b HTTP/1.1" 200 None 2026-03-10 00:57:12,804 - lava_callback - DEBUG - Processing node 69af6c18275c8d3b7939a58b: baseline-arm64-pengutronix 2026-03-10 00:57:12,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6bf0275c8d3b7939a4e6 HTTP/1.1" 200 None 2026-03-10 00:57:12,954 - lava_callback - DEBUG - Processing node 69af6bf0275c8d3b7939a4e6: baseline-nfs-arm64 2026-03-10 00:57:13,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:57:13,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:57:13,930 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:57:13,933 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af6be4275c8d3b7939a4c7/lava-logs.yaml 2026-03-10 00:57:13,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:57:13,972 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:57:13,974 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69af6c18275c8d3b7939a58b/log.txt.gz 2026-03-10 00:57:14,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:57:14,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:57:14,588 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:57:14,590 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af6bf0275c8d3b7939a4e6/log.txt.gz 2026-03-10 00:57:14,623 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:57:14,650 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af6be4275c8d3b7939a4c7/lava_callback.json.gz 2026-03-10 00:57:14,693 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 DEBUG: Setup failed for 69af6be4275c8d3b7939a4c7. Transitting job node result: incomplete -> fail 2026-03-10 00:57:14,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:57:14,811 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69af6c18275c8d3b7939a58b/lava-logs.yaml 2026-03-10 00:57:14,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:57:14,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6be4275c8d3b7939a4c7 HTTP/1.1" 200 None 2026-03-10 00:57:14,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:57:14,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 00:57:14,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:57:15,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af6be4275c8d3b7939a4c7 HTTP/1.1" 200 None 2026-03-10 00:57:15,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:46050 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-10 00:57:15,209 - lava_callback - INFO - Completed processing callback for node 69af6be4275c8d3b7939a4c7 2026-03-10 00:57:15,478 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:57:15,480 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69af6c18275c8d3b7939a58b/lava_callback.json.gz 2026-03-10 00:57:15,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:57:15,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6c18275c8d3b7939a58b HTTP/1.1" 200 None 2026-03-10 00:57:15,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:57:15,553 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:57:15,555 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af6bf0275c8d3b7939a4e6/lava-logs.yaml 2026-03-10 00:57:15,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 00:57:15,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:57:15,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:57:15,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af6c18275c8d3b7939a58b HTTP/1.1" 200 None 2026-03-10 00:57:15,590 - lava_callback - INFO - Completed processing callback for node 69af6c18275c8d3b7939a58b 2026-03-10 00:57:15,742 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:57:15,744 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af6bf0275c8d3b7939a4e6/lava_callback.json.gz DEBUG: Setup failed for 69af6bf0275c8d3b7939a4e6. Transitting job node result: incomplete -> fail 2026-03-10 00:57:15,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:57:15,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6bf0275c8d3b7939a4e6 HTTP/1.1" 200 None 2026-03-10 00:57:15,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:57:15,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 00:57:15,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:57:15,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af6bf0275c8d3b7939a4e6 HTTP/1.1" 200 None 2026-03-10 00:57:16,078 - lava_callback - INFO - Completed processing callback for node 69af6bf0275c8d3b7939a4e6 INFO: 172.20.0.1:46066 - "GET / HTTP/1.0" 200 OK 2026-03-10 00:57:21,669 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:57:21,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-10 00:57:22,262 - lava_callback - INFO - Received callback for node 69af6be7275c8d3b7939a4cd INFO: 172.20.0.1:42284 - "POST /node/69af6be7275c8d3b7939a4cd HTTP/1.0" 202 Accepted 2026-03-10 00:57:22,269 - lava_callback - INFO - Starting async job submit for node 69af6be7275c8d3b7939a4cd 2026-03-10 00:57:22,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:57:22,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6be7275c8d3b7939a4cd HTTP/1.1" 200 None 2026-03-10 00:57:22,361 - lava_callback - DEBUG - Processing node 69af6be7275c8d3b7939a4cd: baseline-nfs-arm64 2026-03-10 00:57:23,129 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:57:23,439 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:57:23,441 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af6be7275c8d3b7939a4cd/log.txt.gz 2026-03-10 00:57:23,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:57:24,254 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:57:24,256 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af6be7275c8d3b7939a4cd/lava-logs.yaml 2026-03-10 00:57:24,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:57:24,460 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:57:24,462 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af6be7275c8d3b7939a4cd/lava_callback.json.gz DEBUG: Setup failed for 69af6be7275c8d3b7939a4cd. Transitting job node result: incomplete -> fail 2026-03-10 00:57:24,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:57:24,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6be7275c8d3b7939a4cd HTTP/1.1" 200 None 2026-03-10 00:57:24,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:57:24,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 00:57:24,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:57:24,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af6be7275c8d3b7939a4cd HTTP/1.1" 200 None 2026-03-10 00:57:24,794 - lava_callback - INFO - Completed processing callback for node 69af6be7275c8d3b7939a4cd INFO: 172.20.0.1:42298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42110 - "GET / HTTP/1.0" 200 OK 2026-03-10 00:57:48,295 - lava_callback - INFO - Received callback for node 69af6c1a275c8d3b7939a58e INFO: 172.20.0.1:42118 - "POST /node/69af6c1a275c8d3b7939a58e HTTP/1.0" 202 Accepted 2026-03-10 00:57:48,301 - lava_callback - INFO - Starting async job submit for node 69af6c1a275c8d3b7939a58e 2026-03-10 00:57:48,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:57:48,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6c1a275c8d3b7939a58e HTTP/1.1" 200 None 2026-03-10 00:57:48,350 - lava_callback - DEBUG - Processing node 69af6c1a275c8d3b7939a58e: baseline-arm64-foundriesio 2026-03-10 00:57:48,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:57:48,719 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:57:48,720 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69af6c1a275c8d3b7939a58e/log.txt.gz 2026-03-10 00:57:48,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:57:49,098 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:57:49,100 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69af6c1a275c8d3b7939a58e/lava-logs.yaml 2026-03-10 00:57:49,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:57:49,287 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:57:49,289 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69af6c1a275c8d3b7939a58e/lava_callback.json.gz DEBUG: Setup failed for 69af6c1a275c8d3b7939a58e. Transitting job node result: incomplete -> fail 2026-03-10 00:57:49,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:57:49,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6c1a275c8d3b7939a58e HTTP/1.1" 200 None 2026-03-10 00:57:49,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:57:49,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 00:57:49,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:57:49,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af6c1a275c8d3b7939a58e HTTP/1.1" 200 None 2026-03-10 00:57:49,535 - lava_callback - INFO - Completed processing callback for node 69af6c1a275c8d3b7939a58e 2026-03-10 00:57:51,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:57:51,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:52808 - "GET / HTTP/1.0" 200 OK 2026-03-10 00:58:00,574 - lava_callback - INFO - Received callback for node 69af6bca275c8d3b7939a46c INFO: 172.20.0.1:34036 - "POST /node/69af6bca275c8d3b7939a46c HTTP/1.0" 202 Accepted 2026-03-10 00:58:00,580 - lava_callback - INFO - Starting async job submit for node 69af6bca275c8d3b7939a46c 2026-03-10 00:58:00,611 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:58:00,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6bca275c8d3b7939a46c HTTP/1.1" 200 None 2026-03-10 00:58:00,627 - lava_callback - DEBUG - Processing node 69af6bca275c8d3b7939a46c: baseline-arm64 2026-03-10 00:58:01,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:58:01,173 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:58:01,174 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af6bca275c8d3b7939a46c/log.txt.gz 2026-03-10 00:58:01,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:58:01,619 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:58:01,621 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af6bca275c8d3b7939a46c/lava-logs.yaml 2026-03-10 00:58:01,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:58:01,659 - lava_callback - INFO - Received callback for node 69af6bfe275c8d3b7939a51d INFO: 172.20.0.1:34040 - "POST /node/69af6bfe275c8d3b7939a51d HTTP/1.0" 202 Accepted 2026-03-10 00:58:01,668 - lava_callback - INFO - Starting async job submit for node 69af6bfe275c8d3b7939a51d 2026-03-10 00:58:01,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:58:01,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6bfe275c8d3b7939a51d HTTP/1.1" 200 None 2026-03-10 00:58:01,762 - lava_callback - DEBUG - Processing node 69af6bfe275c8d3b7939a51d: baseline-arm64-broonie 2026-03-10 00:58:01,783 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:58:01,825 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af6bca275c8d3b7939a46c/lava_callback.json.gz DEBUG: Setup failed for 69af6bca275c8d3b7939a46c. Transitting job node result: incomplete -> fail 2026-03-10 00:58:01,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:58:02,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6bca275c8d3b7939a46c HTTP/1.1" 200 None 2026-03-10 00:58:02,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:58:02,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:58:02,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 00:58:02,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:58:02,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af6bca275c8d3b7939a46c HTTP/1.1" 200 None 2026-03-10 00:58:02,229 - lava_callback - INFO - Completed processing callback for node 69af6bca275c8d3b7939a46c 2026-03-10 00:58:02,250 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:58:02,252 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af6bfe275c8d3b7939a51d/log.txt.gz 2026-03-10 00:58:02,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:58:02,675 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:58:02,677 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af6bfe275c8d3b7939a51d/lava-logs.yaml 2026-03-10 00:58:02,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:58:02,837 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:58:02,840 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af6bfe275c8d3b7939a51d/lava_callback.json.gz DEBUG: Setup failed for 69af6bfe275c8d3b7939a51d. Transitting job node result: incomplete -> fail 2026-03-10 00:58:02,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:58:02,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6bfe275c8d3b7939a51d HTTP/1.1" 200 None 2026-03-10 00:58:02,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:58:02,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 00:58:02,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:58:02,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af6bfe275c8d3b7939a51d HTTP/1.1" 200 None 2026-03-10 00:58:03,072 - lava_callback - INFO - Completed processing callback for node 69af6bfe275c8d3b7939a51d INFO: 172.20.0.1:34046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34052 - "GET / HTTP/1.0" 200 OK 2026-03-10 00:58:13,955 - lava_callback - INFO - Received callback for node 69af6bdf275c8d3b7939a4b6 INFO: 172.20.0.1:47250 - "POST /node/69af6bdf275c8d3b7939a4b6 HTTP/1.0" 202 Accepted 2026-03-10 00:58:13,961 - lava_callback - INFO - Starting async job submit for node 69af6bdf275c8d3b7939a4b6 2026-03-10 00:58:13,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:58:14,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6bdf275c8d3b7939a4b6 HTTP/1.1" 200 None 2026-03-10 00:58:14,014 - lava_callback - DEBUG - Processing node 69af6bdf275c8d3b7939a4b6: baseline-nfs-arm64 2026-03-10 00:58:14,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:58:14,535 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:58:14,536 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af6bdf275c8d3b7939a4b6/log.txt.gz 2026-03-10 00:58:14,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:58:14,960 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:58:14,962 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af6bdf275c8d3b7939a4b6/lava-logs.yaml 2026-03-10 00:58:14,971 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:58:15,116 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:58:15,117 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af6bdf275c8d3b7939a4b6/lava_callback.json.gz DEBUG: Setup failed for 69af6bdf275c8d3b7939a4b6. Transitting job node result: incomplete -> fail INFO: 172.20.0.1:47266 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-10 00:58:15,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:58:15,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6bdf275c8d3b7939a4b6 HTTP/1.1" 200 None 2026-03-10 00:58:15,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:58:15,220 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 00:58:15,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:58:15,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af6bdf275c8d3b7939a4b6 HTTP/1.1" 200 None 2026-03-10 00:58:15,316 - lava_callback - INFO - Completed processing callback for node 69af6bdf275c8d3b7939a4b6 INFO: 172.20.0.1:47276 - "GET / HTTP/1.0" 200 OK 2026-03-10 00:58:21,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:58:21,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:34480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41182 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33052 - "GET / HTTP/1.0" 200 OK 2026-03-10 00:59:47,309 - lava_callback - INFO - Received callback for node 69af6c0a275c8d3b7939a545 INFO: 172.20.0.1:33054 - "POST /node/69af6c0a275c8d3b7939a545 HTTP/1.0" 202 Accepted 2026-03-10 00:59:47,316 - lava_callback - INFO - Starting async job submit for node 69af6c0a275c8d3b7939a545 2026-03-10 00:59:47,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:59:47,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6c0a275c8d3b7939a545 HTTP/1.1" 200 None 2026-03-10 00:59:47,367 - lava_callback - DEBUG - Processing node 69af6c0a275c8d3b7939a545: baseline-arm64-broonie 2026-03-10 00:59:47,819 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:59:47,974 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:59:47,976 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af6c0a275c8d3b7939a545/log.txt.gz 2026-03-10 00:59:48,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:59:48,439 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:59:48,441 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af6c0a275c8d3b7939a545/lava-logs.yaml 2026-03-10 00:59:48,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 00:59:48,600 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 00:59:48,602 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af6c0a275c8d3b7939a545/lava_callback.json.gz DEBUG: Setup failed for 69af6c0a275c8d3b7939a545. Transitting job node result: incomplete -> fail 2026-03-10 00:59:48,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:59:48,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6c0a275c8d3b7939a545 HTTP/1.1" 200 None 2026-03-10 00:59:48,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:59:48,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 00:59:48,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:59:48,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af6c0a275c8d3b7939a545 HTTP/1.1" 200 None 2026-03-10 00:59:48,802 - lava_callback - INFO - Completed processing callback for node 69af6c0a275c8d3b7939a545 2026-03-10 00:59:51,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 00:59:51,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:48244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47340 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:00:13,677 - lava_callback - INFO - Received callback for node 69af6d17275c8d3b7939a671 INFO: 172.20.0.1:46890 - "POST /node/69af6d17275c8d3b7939a671 HTTP/1.0" 202 Accepted 2026-03-10 01:00:13,683 - lava_callback - INFO - Starting async job submit for node 69af6d17275c8d3b7939a671 2026-03-10 01:00:13,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:00:13,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6d17275c8d3b7939a671 HTTP/1.1" 200 None 2026-03-10 01:00:13,704 - lava_callback - DEBUG - Processing node 69af6d17275c8d3b7939a671: baseline-arm64 2026-03-10 01:00:13,708 - lava_callback - WARNING - Failed to upload log 2026-03-10 01:00:13,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:00:13,870 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:00:13,873 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af6d17275c8d3b7939a671/lava-logs.yaml 2026-03-10 01:00:13,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:00:14,007 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:00:14,009 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af6d17275c8d3b7939a671/lava_callback.json.gz 2026-03-10 01:00:14,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:00:14,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6d17275c8d3b7939a671 HTTP/1.1" 200 None 2026-03-10 01:00:14,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:00:14,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 01:00:14,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:00:14,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af6d17275c8d3b7939a671 HTTP/1.1" 200 None 2026-03-10 01:00:14,069 - lava_callback - INFO - Completed processing callback for node 69af6d17275c8d3b7939a671 INFO: 172.20.0.1:46896 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46904 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:00:21,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:00:21,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:48348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42934 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:00:47,105 - lava_callback - INFO - Received callback for node 69af6c05275c8d3b7939a53a INFO: 172.20.0.1:42940 - "POST /node/69af6c05275c8d3b7939a53a HTTP/1.0" 202 Accepted 2026-03-10 01:00:47,117 - lava_callback - INFO - Starting async job submit for node 69af6c05275c8d3b7939a53a 2026-03-10 01:00:47,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:00:47,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6c05275c8d3b7939a53a HTTP/1.1" 200 None 2026-03-10 01:00:47,214 - lava_callback - DEBUG - Processing node 69af6c05275c8d3b7939a53a: baseline-arm64-broonie 2026-03-10 01:00:48,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:00:48,477 - lava_callback - INFO - Received callback for node 69af6d2a275c8d3b7939a67a INFO: 172.20.0.1:42946 - "POST /node/69af6d2a275c8d3b7939a67a HTTP/1.0" 202 Accepted 2026-03-10 01:00:48,487 - lava_callback - INFO - Starting async job submit for node 69af6d2a275c8d3b7939a67a 2026-03-10 01:00:48,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:00:48,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6d2a275c8d3b7939a67a HTTP/1.1" 200 None 2026-03-10 01:00:48,567 - lava_callback - DEBUG - Processing node 69af6d2a275c8d3b7939a67a: baseline-x86 2026-03-10 01:00:48,598 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:00:48,600 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af6c05275c8d3b7939a53a/log.txt.gz 2026-03-10 01:00:49,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:00:49,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:00:49,561 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:00:49,563 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69af6d2a275c8d3b7939a67a/log.txt.gz 2026-03-10 01:00:49,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:00:50,117 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:00:50,120 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af6c05275c8d3b7939a53a/lava-logs.yaml 2026-03-10 01:00:50,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:00:50,348 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:00:50,351 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69af6d2a275c8d3b7939a67a/lava-logs.yaml 2026-03-10 01:00:50,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:00:50,627 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:00:50,629 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af6c05275c8d3b7939a53a/lava_callback.json.gz DEBUG: Setup failed for 69af6c05275c8d3b7939a53a. Transitting job node result: incomplete -> fail 2026-03-10 01:00:50,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:00:50,765 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:00:50,768 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69af6d2a275c8d3b7939a67a/lava_callback.json.gz 2026-03-10 01:00:50,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:00:50,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6c05275c8d3b7939a53a HTTP/1.1" 200 None 2026-03-10 01:00:50,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:00:50,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6d2a275c8d3b7939a67a HTTP/1.1" 200 None 2026-03-10 01:00:50,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:00:50,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 01:00:50,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:00:50,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af635a275c8d3b79399ddd HTTP/1.1" 200 None 2026-03-10 01:00:50,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:00:50,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af6c05275c8d3b7939a53a HTTP/1.1" 200 None 2026-03-10 01:00:50,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af6d2a275c8d3b7939a67a HTTP/1.1" 200 None 2026-03-10 01:00:50,968 - lava_callback - INFO - Completed processing callback for node 69af6d2a275c8d3b7939a67a 2026-03-10 01:00:51,097 - lava_callback - INFO - Completed processing callback for node 69af6c05275c8d3b7939a53a 2026-03-10 01:00:51,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:00:51,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:44474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36054 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:01:06,872 - lava_callback - INFO - Received callback for node 69af6c1b275c8d3b7939a591 INFO: 172.20.0.1:36066 - "POST /node/69af6c1b275c8d3b7939a591 HTTP/1.0" 202 Accepted 2026-03-10 01:01:06,884 - lava_callback - INFO - Starting async job submit for node 69af6c1b275c8d3b7939a591 2026-03-10 01:01:06,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:01:06,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6c1b275c8d3b7939a591 HTTP/1.1" 200 None 2026-03-10 01:01:06,956 - lava_callback - DEBUG - Processing node 69af6c1b275c8d3b7939a591: baseline-arm64-foundriesio 2026-03-10 01:01:07,045 - lava_callback - WARNING - Failed to upload log 2026-03-10 01:01:07,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:01:07,545 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:01:07,548 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69af6c1b275c8d3b7939a591/lava-logs.yaml 2026-03-10 01:01:07,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:01:07,928 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:01:07,930 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69af6c1b275c8d3b7939a591/lava_callback.json.gz 2026-03-10 01:01:07,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:01:07,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6c1b275c8d3b7939a591 HTTP/1.1" 200 None 2026-03-10 01:01:07,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:01:08,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 01:01:08,015 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:01:08,052 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af6c1b275c8d3b7939a591 HTTP/1.1" 200 None 2026-03-10 01:01:08,175 - lava_callback - INFO - Completed processing callback for node 69af6c1b275c8d3b7939a591 INFO: 172.20.0.1:59194 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59204 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:01:19,989 - lava_callback - INFO - Received callback for node 69af6d2d275c8d3b7939a683 INFO: 172.20.0.1:38294 - "POST /node/69af6d2d275c8d3b7939a683 HTTP/1.0" 202 Accepted 2026-03-10 01:01:19,999 - lava_callback - INFO - Starting async job submit for node 69af6d2d275c8d3b7939a683 2026-03-10 01:01:20,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:01:20,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6d2d275c8d3b7939a683 HTTP/1.1" 200 None 2026-03-10 01:01:20,070 - lava_callback - DEBUG - Processing node 69af6d2d275c8d3b7939a683: baseline-x86 2026-03-10 01:01:20,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:01:20,700 - lava_callback - INFO - Received callback for node 69af6c00275c8d3b7939a525 INFO: 172.20.0.1:38302 - "POST /node/69af6c00275c8d3b7939a525 HTTP/1.0" 202 Accepted 2026-03-10 01:01:20,734 - lava_callback - INFO - Starting async job submit for node 69af6c00275c8d3b7939a525 2026-03-10 01:01:20,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:01:20,825 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:01:20,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6c00275c8d3b7939a525 HTTP/1.1" 200 None 2026-03-10 01:01:20,832 - lava_callback - DEBUG - Processing node 69af6c00275c8d3b7939a525: baseline-arm64-broonie 2026-03-10 01:01:20,833 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69af6d2d275c8d3b7939a683/log.txt.gz 2026-03-10 01:01:21,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:01:21,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:01:21,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:01:21,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-10 01:01:21,860 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:01:21,863 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69af6d2d275c8d3b7939a683/lava-logs.yaml 2026-03-10 01:01:21,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:01:21,895 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:01:21,898 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af6c00275c8d3b7939a525/log.txt.gz 2026-03-10 01:01:22,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:01:22,364 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:01:22,366 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69af6d2d275c8d3b7939a683/lava_callback.json.gz 2026-03-10 01:01:22,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:01:22,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6d2d275c8d3b7939a683 HTTP/1.1" 200 None 2026-03-10 01:01:22,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:01:22,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af635a275c8d3b79399ddd HTTP/1.1" 200 None 2026-03-10 01:01:22,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:01:22,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af6d2d275c8d3b7939a683 HTTP/1.1" 200 None 2026-03-10 01:01:22,513 - lava_callback - INFO - Completed processing callback for node 69af6d2d275c8d3b7939a683 2026-03-10 01:01:22,793 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:01:22,796 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af6c00275c8d3b7939a525/lava-logs.yaml 2026-03-10 01:01:22,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:01:23,326 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:01:23,329 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af6c00275c8d3b7939a525/lava_callback.json.gz DEBUG: Setup failed for 69af6c00275c8d3b7939a525. Transitting job node result: incomplete -> fail 2026-03-10 01:01:23,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:01:23,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6c00275c8d3b7939a525 HTTP/1.1" 200 None 2026-03-10 01:01:23,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:01:23,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 01:01:23,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:01:23,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af6c00275c8d3b7939a525 HTTP/1.1" 200 None 2026-03-10 01:01:23,704 - lava_callback - INFO - Completed processing callback for node 69af6c00275c8d3b7939a525 INFO: 172.20.0.1:38306 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:01:31,154 - lava_callback - INFO - Received callback for node 69acca62bcd207e915d77496 INFO: 172.20.0.1:39862 - "POST /node/69acca62bcd207e915d77496 HTTP/1.0" 202 Accepted 2026-03-10 01:01:31,163 - lava_callback - INFO - Starting async job submit for node 69acca62bcd207e915d77496 2026-03-10 01:01:31,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:01:31,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acca62bcd207e915d77496 HTTP/1.1" 200 None 2026-03-10 01:01:31,190 - lava_callback - DEBUG - Processing node 69acca62bcd207e915d77496: baseline-arm64-baylibre 2026-03-10 01:01:31,190 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acca62bcd207e915d77496 2026-03-10 01:01:31,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:01:31,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acca62bcd207e915d77496 HTTP/1.1" 200 None 2026-03-10 01:01:31,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:01:31,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa1d4a8c4d487a0e751161 HTTP/1.1" 200 None 2026-03-10 01:01:31,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:01:31,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acca62bcd207e915d77496 HTTP/1.1" 200 None 2026-03-10 01:01:31,307 - lava_callback - ERROR - Error processing callback for node 69acca62bcd207e915d77496: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-10 01:01:34,273 - lava_callback - INFO - Received callback for node 69af6d66275c8d3b7939a6b3 INFO: 172.20.0.1:39878 - "POST /node/69af6d66275c8d3b7939a6b3 HTTP/1.0" 202 Accepted 2026-03-10 01:01:34,286 - lava_callback - INFO - Starting async job submit for node 69af6d66275c8d3b7939a6b3 2026-03-10 01:01:34,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:01:34,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6d66275c8d3b7939a6b3 HTTP/1.1" 200 None 2026-03-10 01:01:34,371 - lava_callback - DEBUG - Processing node 69af6d66275c8d3b7939a6b3: baseline-arm64 2026-03-10 01:01:34,379 - lava_callback - WARNING - Failed to upload log 2026-03-10 01:01:34,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:01:34,720 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:01:34,722 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af6d66275c8d3b7939a6b3/lava-logs.yaml 2026-03-10 01:01:34,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:01:35,046 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:01:35,048 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af6d66275c8d3b7939a6b3/lava_callback.json.gz 2026-03-10 01:01:35,054 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:01:35,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6d66275c8d3b7939a6b3 HTTP/1.1" 200 None 2026-03-10 01:01:35,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:01:35,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 01:01:35,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:01:35,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af6d66275c8d3b7939a6b3 HTTP/1.1" 200 None 2026-03-10 01:01:35,168 - lava_callback - INFO - Completed processing callback for node 69af6d66275c8d3b7939a6b3 INFO: 172.20.0.1:39886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50822 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:01:51,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:01:51,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:33894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49574 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49588 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:02:18,274 - lava_callback - INFO - Received callback for node 69af6d2b275c8d3b7939a67d INFO: 172.20.0.1:49594 - "POST /node/69af6d2b275c8d3b7939a67d HTTP/1.0" 202 Accepted 2026-03-10 01:02:18,280 - lava_callback - INFO - Starting async job submit for node 69af6d2b275c8d3b7939a67d 2026-03-10 01:02:18,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:02:18,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6d2b275c8d3b7939a67d HTTP/1.1" 200 None 2026-03-10 01:02:18,314 - lava_callback - DEBUG - Processing node 69af6d2b275c8d3b7939a67d: baseline-x86 2026-03-10 01:02:18,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:02:18,755 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:02:18,757 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69af6d2b275c8d3b7939a67d/log.txt.gz 2026-03-10 01:02:18,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:02:19,193 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:02:19,195 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69af6d2b275c8d3b7939a67d/lava-logs.yaml 2026-03-10 01:02:19,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:02:19,417 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:02:19,418 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69af6d2b275c8d3b7939a67d/lava_callback.json.gz 2026-03-10 01:02:19,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:02:19,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6d2b275c8d3b7939a67d HTTP/1.1" 200 None 2026-03-10 01:02:19,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:02:19,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af635a275c8d3b79399ddd HTTP/1.1" 200 None 2026-03-10 01:02:19,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:02:19,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af6d2b275c8d3b7939a67d HTTP/1.1" 200 None 2026-03-10 01:02:19,495 - lava_callback - INFO - Completed processing callback for node 69af6d2b275c8d3b7939a67d 2026-03-10 01:02:21,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:02:21,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-10 01:02:25,980 - lava_callback - INFO - Received callback for node 69af6bce275c8d3b7939a478 INFO: 172.20.0.1:35340 - "POST /node/69af6bce275c8d3b7939a478 HTTP/1.0" 202 Accepted 2026-03-10 01:02:25,986 - lava_callback - INFO - Starting async job submit for node 69af6bce275c8d3b7939a478 2026-03-10 01:02:26,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:02:26,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6bce275c8d3b7939a478 HTTP/1.1" 200 None 2026-03-10 01:02:26,018 - lava_callback - DEBUG - Processing node 69af6bce275c8d3b7939a478: baseline-arm64 2026-03-10 01:02:26,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:02:26,481 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:02:26,483 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af6bce275c8d3b7939a478/log.txt.gz INFO: 172.20.0.1:35348 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:02:26,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:02:26,863 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:02:26,865 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af6bce275c8d3b7939a478/lava-logs.yaml 2026-03-10 01:02:26,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:02:27,033 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:02:27,035 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af6bce275c8d3b7939a478/lava_callback.json.gz 2026-03-10 01:02:27,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:02:27,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6bce275c8d3b7939a478 HTTP/1.1" 200 None 2026-03-10 01:02:27,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:02:27,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 01:02:27,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:02:27,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af6bce275c8d3b7939a478 HTTP/1.1" 200 None 2026-03-10 01:02:27,114 - lava_callback - INFO - Completed processing callback for node 69af6bce275c8d3b7939a478 INFO: 172.20.0.1:35410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53472 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:02:51,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:02:51,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:42166 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:02:59,525 - lava_callback - INFO - Received callback for node 69accab9bcd207e915d774cb INFO: 172.20.0.1:42178 - "POST /node/69accab9bcd207e915d774cb HTTP/1.0" 202 Accepted 2026-03-10 01:02:59,530 - lava_callback - INFO - Starting async job submit for node 69accab9bcd207e915d774cb 2026-03-10 01:02:59,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:02:59,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69accab9bcd207e915d774cb HTTP/1.1" 200 None 2026-03-10 01:02:59,546 - lava_callback - DEBUG - Processing node 69accab9bcd207e915d774cb: baseline-arm64-clabbe 2026-03-10 01:02:59,546 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69accab9bcd207e915d774cb 2026-03-10 01:02:59,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:02:59,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69accab9bcd207e915d774cb HTTP/1.1" 200 None 2026-03-10 01:02:59,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:02:59,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa1d4a8c4d487a0e751161 HTTP/1.1" 200 None 2026-03-10 01:02:59,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:02:59,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69accab9bcd207e915d774cb HTTP/1.1" 200 None 2026-03-10 01:02:59,598 - lava_callback - ERROR - Error processing callback for node 69accab9bcd207e915d774cb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42724 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40752 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40766 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:04:19,504 - lava_callback - INFO - Received callback for node 69accb09bcd207e915d774df INFO: 172.20.0.1:40782 - "POST /node/69accb09bcd207e915d774df HTTP/1.0" 202 Accepted 2026-03-10 01:04:19,509 - lava_callback - INFO - Starting async job submit for node 69accb09bcd207e915d774df 2026-03-10 01:04:19,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:04:19,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69accb09bcd207e915d774df HTTP/1.1" 200 None 2026-03-10 01:04:19,525 - lava_callback - DEBUG - Processing node 69accb09bcd207e915d774df: baseline-arm64-clabbe 2026-03-10 01:04:19,526 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69accb09bcd207e915d774df 2026-03-10 01:04:19,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:04:19,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69accb09bcd207e915d774df HTTP/1.1" 200 None 2026-03-10 01:04:19,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:04:19,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa1d4a8c4d487a0e751161 HTTP/1.1" 200 None 2026-03-10 01:04:19,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:04:19,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69accb09bcd207e915d774df HTTP/1.1" 200 None 2026-03-10 01:04:19,613 - lava_callback - ERROR - Error processing callback for node 69accb09bcd207e915d774df: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55322 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55332 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:05:25,289 - lava_callback - INFO - Received callback for node 69af6be1275c8d3b7939a4bd INFO: 172.20.0.1:46992 - "POST /node/69af6be1275c8d3b7939a4bd HTTP/1.0" 202 Accepted 2026-03-10 01:05:25,295 - lava_callback - INFO - Starting async job submit for node 69af6be1275c8d3b7939a4bd 2026-03-10 01:05:25,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:05:25,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6be1275c8d3b7939a4bd HTTP/1.1" 200 None 2026-03-10 01:05:25,340 - lava_callback - DEBUG - Processing node 69af6be1275c8d3b7939a4bd: baseline-nfs-arm64 2026-03-10 01:05:25,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:05:25,945 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:05:25,947 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af6be1275c8d3b7939a4bd/log.txt.gz 2026-03-10 01:05:26,186 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:05:26,415 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:05:26,417 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af6be1275c8d3b7939a4bd/lava-logs.yaml 2026-03-10 01:05:26,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:47002 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:05:26,595 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:05:26,597 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af6be1275c8d3b7939a4bd/lava_callback.json.gz 2026-03-10 01:05:26,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:05:26,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6be1275c8d3b7939a4bd HTTP/1.1" 200 None 2026-03-10 01:05:26,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:05:26,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 01:05:26,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:05:26,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af6be1275c8d3b7939a4bd HTTP/1.1" 200 None 2026-03-10 01:05:26,674 - lava_callback - INFO - Completed processing callback for node 69af6be1275c8d3b7939a4bd INFO: 172.20.0.1:36446 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:05:36,657 - lava_callback - INFO - Received callback for node 69af6e4d275c8d3b7939a6fd INFO: 172.20.0.1:36450 - "POST /node/69af6e4d275c8d3b7939a6fd HTTP/1.0" 202 Accepted 2026-03-10 01:05:36,665 - lava_callback - INFO - Starting async job submit for node 69af6e4d275c8d3b7939a6fd 2026-03-10 01:05:36,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:05:36,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6e4d275c8d3b7939a6fd HTTP/1.1" 200 None 2026-03-10 01:05:36,695 - lava_callback - DEBUG - Processing node 69af6e4d275c8d3b7939a6fd: baseline-arm64 2026-03-10 01:05:36,744 - lava_callback - WARNING - Failed to upload log 2026-03-10 01:05:36,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:05:36,906 - lava_callback - INFO - Received callback for node 69af6bec275c8d3b7939a4d8 INFO: 172.20.0.1:36460 - "POST /node/69af6bec275c8d3b7939a4d8 HTTP/1.0" 202 Accepted 2026-03-10 01:05:36,913 - lava_callback - INFO - Starting async job submit for node 69af6bec275c8d3b7939a4d8 2026-03-10 01:05:36,924 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:05:36,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:05:36,942 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af6e4d275c8d3b7939a6fd/lava-logs.yaml 2026-03-10 01:05:36,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:05:36,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6bec275c8d3b7939a4d8 HTTP/1.1" 200 None 2026-03-10 01:05:36,976 - lava_callback - DEBUG - Processing node 69af6bec275c8d3b7939a4d8: baseline-nfs-arm64 2026-03-10 01:05:37,239 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:05:37,287 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af6e4d275c8d3b7939a6fd/lava_callback.json.gz 2026-03-10 01:05:37,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:05:37,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6e4d275c8d3b7939a6fd HTTP/1.1" 200 None 2026-03-10 01:05:37,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:05:37,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 01:05:37,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:05:37,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:05:37,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af6e4d275c8d3b7939a6fd HTTP/1.1" 200 None 2026-03-10 01:05:37,663 - lava_callback - INFO - Completed processing callback for node 69af6e4d275c8d3b7939a6fd 2026-03-10 01:05:37,823 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:05:37,825 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af6bec275c8d3b7939a4d8/log.txt.gz 2026-03-10 01:05:38,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:05:38,271 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:05:38,274 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af6bec275c8d3b7939a4d8/lava-logs.yaml 2026-03-10 01:05:38,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:05:38,445 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:05:38,446 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af6bec275c8d3b7939a4d8/lava_callback.json.gz 2026-03-10 01:05:38,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:05:38,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6bec275c8d3b7939a4d8 HTTP/1.1" 200 None 2026-03-10 01:05:38,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:05:38,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 01:05:38,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:05:38,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af6bec275c8d3b7939a4d8 HTTP/1.1" 200 None 2026-03-10 01:05:38,626 - lava_callback - INFO - Completed processing callback for node 69af6bec275c8d3b7939a4d8 INFO: 172.20.0.1:59150 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:05:51,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:05:51,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:58722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43618 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:06:13,854 - lava_callback - INFO - Received callback for node 69af6bcf275c8d3b7939a47b INFO: 172.20.0.1:34424 - "POST /node/69af6bcf275c8d3b7939a47b HTTP/1.0" 202 Accepted 2026-03-10 01:06:13,860 - lava_callback - INFO - Starting async job submit for node 69af6bcf275c8d3b7939a47b 2026-03-10 01:06:13,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:06:13,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6bcf275c8d3b7939a47b HTTP/1.1" 200 None 2026-03-10 01:06:13,907 - lava_callback - DEBUG - Processing node 69af6bcf275c8d3b7939a47b: baseline-arm64 2026-03-10 01:06:14,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:06:14,319 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:06:14,321 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af6bcf275c8d3b7939a47b/log.txt.gz 2026-03-10 01:06:14,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:06:14,704 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:06:14,706 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af6bcf275c8d3b7939a47b/lava-logs.yaml 2026-03-10 01:06:14,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:06:14,839 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:06:14,841 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af6bcf275c8d3b7939a47b/lava_callback.json.gz 2026-03-10 01:06:14,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:06:14,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6bcf275c8d3b7939a47b HTTP/1.1" 200 None 2026-03-10 01:06:14,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:06:14,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 01:06:14,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:06:14,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af6bcf275c8d3b7939a47b HTTP/1.1" 200 None 2026-03-10 01:06:15,003 - lava_callback - INFO - Completed processing callback for node 69af6bcf275c8d3b7939a47b INFO: 172.20.0.1:34432 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34438 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:06:21,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:06:21,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:35648 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:06:27,383 - lava_callback - INFO - Received callback for node 69af6be2275c8d3b7939a4c4 INFO: 172.20.0.1:35656 - "POST /node/69af6be2275c8d3b7939a4c4 HTTP/1.0" 202 Accepted 2026-03-10 01:06:27,390 - lava_callback - INFO - Starting async job submit for node 69af6be2275c8d3b7939a4c4 2026-03-10 01:06:27,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:06:27,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6be2275c8d3b7939a4c4 HTTP/1.1" 200 None 2026-03-10 01:06:27,439 - lava_callback - DEBUG - Processing node 69af6be2275c8d3b7939a4c4: baseline-nfs-arm64 2026-03-10 01:06:27,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:06:27,890 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:06:27,892 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af6be2275c8d3b7939a4c4/log.txt.gz 2026-03-10 01:06:28,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:06:28,300 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:06:28,302 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af6be2275c8d3b7939a4c4/lava-logs.yaml 2026-03-10 01:06:28,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:06:28,456 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:06:28,458 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af6be2275c8d3b7939a4c4/lava_callback.json.gz 2026-03-10 01:06:28,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:06:28,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6be2275c8d3b7939a4c4 HTTP/1.1" 200 None 2026-03-10 01:06:28,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:06:28,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 01:06:28,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:06:28,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af6be2275c8d3b7939a4c4 HTTP/1.1" 200 None 2026-03-10 01:06:28,621 - lava_callback - INFO - Completed processing callback for node 69af6be2275c8d3b7939a4c4 INFO: 172.20.0.1:52946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48974 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:06:51,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:06:51,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:60484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48412 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:07:06,723 - lava_callback - INFO - Received callback for node 69af6e9e275c8d3b7939a71a INFO: 172.20.0.1:48428 - "POST /node/69af6e9e275c8d3b7939a71a HTTP/1.0" 202 Accepted 2026-03-10 01:07:06,728 - lava_callback - INFO - Starting async job submit for node 69af6e9e275c8d3b7939a71a 2026-03-10 01:07:06,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:07:06,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6e9e275c8d3b7939a71a HTTP/1.1" 200 None 2026-03-10 01:07:06,753 - lava_callback - DEBUG - Processing node 69af6e9e275c8d3b7939a71a: baseline-arm64 2026-03-10 01:07:06,758 - lava_callback - WARNING - Failed to upload log 2026-03-10 01:07:06,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:07:06,897 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:07:06,898 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af6e9e275c8d3b7939a71a/lava-logs.yaml 2026-03-10 01:07:06,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:07:07,027 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:07:07,029 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af6e9e275c8d3b7939a71a/lava_callback.json.gz 2026-03-10 01:07:07,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:07:07,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6e9e275c8d3b7939a71a HTTP/1.1" 200 None 2026-03-10 01:07:07,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:07:07,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 01:07:07,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:07:07,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af6e9e275c8d3b7939a71a HTTP/1.1" 200 None 2026-03-10 01:07:07,117 - lava_callback - INFO - Completed processing callback for node 69af6e9e275c8d3b7939a71a INFO: 172.20.0.1:48568 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48584 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:07:21,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:07:21,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:42948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46486 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39204 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39216 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:09:20,745 - lava_callback - INFO - Received callback for node 69af6c06275c8d3b7939a53e INFO: 172.20.0.1:59090 - "POST /node/69af6c06275c8d3b7939a53e HTTP/1.0" 202 Accepted 2026-03-10 01:09:20,751 - lava_callback - INFO - Starting async job submit for node 69af6c06275c8d3b7939a53e 2026-03-10 01:09:20,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:09:20,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6c06275c8d3b7939a53e HTTP/1.1" 200 None 2026-03-10 01:09:20,800 - lava_callback - DEBUG - Processing node 69af6c06275c8d3b7939a53e: baseline-arm64-broonie 2026-03-10 01:09:21,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:09:21,289 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:09:21,290 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af6c06275c8d3b7939a53e/log.txt.gz 2026-03-10 01:09:21,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:09:21,710 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:09:21,713 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af6c06275c8d3b7939a53e/lava-logs.yaml 2026-03-10 01:09:21,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:09:21,761 - lava_callback - INFO - Received callback for node 69af6bd8275c8d3b7939a4a7 INFO: 172.20.0.1:59092 - "POST /node/69af6bd8275c8d3b7939a4a7 HTTP/1.0" 202 Accepted 2026-03-10 01:09:21,771 - lava_callback - INFO - Starting async job submit for node 69af6bd8275c8d3b7939a4a7 2026-03-10 01:09:21,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:09:21,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6bd8275c8d3b7939a4a7 HTTP/1.1" 200 None 2026-03-10 01:09:21,813 - lava_callback - DEBUG - Processing node 69af6bd8275c8d3b7939a4a7: baseline-arm64 2026-03-10 01:09:21,861 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:09:21,914 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af6c06275c8d3b7939a53e/lava_callback.json.gz DEBUG: Setup failed for 69af6c06275c8d3b7939a53e. Transitting job node result: incomplete -> fail 2026-03-10 01:09:22,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:09:22,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6c06275c8d3b7939a53e HTTP/1.1" 200 None 2026-03-10 01:09:22,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:09:22,254 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 01:09:22,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:09:22,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:09:22,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af6c06275c8d3b7939a53e HTTP/1.1" 200 None 2026-03-10 01:09:22,413 - lava_callback - INFO - Completed processing callback for node 69af6c06275c8d3b7939a53e 2026-03-10 01:09:22,480 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:09:22,482 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af6bd8275c8d3b7939a4a7/log.txt.gz 2026-03-10 01:09:22,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:09:23,049 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:09:23,052 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af6bd8275c8d3b7939a4a7/lava-logs.yaml 2026-03-10 01:09:23,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:09:23,192 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:09:23,194 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af6bd8275c8d3b7939a4a7/lava_callback.json.gz 2026-03-10 01:09:23,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:09:23,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6bd8275c8d3b7939a4a7 HTTP/1.1" 200 None 2026-03-10 01:09:23,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:09:23,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 01:09:23,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:09:23,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af6bd8275c8d3b7939a4a7 HTTP/1.1" 200 None 2026-03-10 01:09:23,320 - lava_callback - INFO - Completed processing callback for node 69af6bd8275c8d3b7939a4a7 INFO: 172.20.0.1:59102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55294 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:09:52,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:09:52,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-10 01:09:52,929 - lava_callback - INFO - Received callback for node 69af6f34275c8d3b7939a737 INFO: 172.20.0.1:46834 - "POST /node/69af6f34275c8d3b7939a737 HTTP/1.0" 202 Accepted 2026-03-10 01:09:52,935 - lava_callback - INFO - Starting async job submit for node 69af6f34275c8d3b7939a737 2026-03-10 01:09:52,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:09:52,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6f34275c8d3b7939a737 HTTP/1.1" 200 None 2026-03-10 01:09:52,970 - lava_callback - DEBUG - Processing node 69af6f34275c8d3b7939a737: baseline-arm64 2026-03-10 01:09:53,018 - lava_callback - WARNING - Failed to upload log 2026-03-10 01:09:53,045 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:09:53,213 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:09:53,215 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af6f34275c8d3b7939a737/lava-logs.yaml 2026-03-10 01:09:53,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:09:53,347 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:09:53,349 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af6f34275c8d3b7939a737/lava_callback.json.gz 2026-03-10 01:09:53,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:09:53,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6f34275c8d3b7939a737 HTTP/1.1" 200 None 2026-03-10 01:09:53,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:09:53,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 01:09:53,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:09:53,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af6f34275c8d3b7939a737 HTTP/1.1" 200 None 2026-03-10 01:09:53,488 - lava_callback - INFO - Completed processing callback for node 69af6f34275c8d3b7939a737 INFO: 172.20.0.1:46838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56038 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56052 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:10:22,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:10:22,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:36230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52424 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:10:48,631 - lava_callback - INFO - Received callback for node 69af6f87275c8d3b7939a752 INFO: 172.20.0.1:52430 - "POST /node/69af6f87275c8d3b7939a752 HTTP/1.0" 202 Accepted 2026-03-10 01:10:48,636 - lava_callback - INFO - Starting async job submit for node 69af6f87275c8d3b7939a752 2026-03-10 01:10:48,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:10:48,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6f87275c8d3b7939a752 HTTP/1.1" 200 None 2026-03-10 01:10:48,654 - lava_callback - DEBUG - Processing node 69af6f87275c8d3b7939a752: baseline-nfs-arm64 2026-03-10 01:10:48,660 - lava_callback - WARNING - Failed to upload log 2026-03-10 01:10:48,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:10:48,818 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:10:48,820 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af6f87275c8d3b7939a752/lava-logs.yaml 2026-03-10 01:10:48,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:10:48,977 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:10:48,979 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af6f87275c8d3b7939a752/lava_callback.json.gz 2026-03-10 01:10:48,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:10:48,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6f87275c8d3b7939a752 HTTP/1.1" 200 None 2026-03-10 01:10:48,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:10:49,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 01:10:49,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:10:49,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af6f87275c8d3b7939a752 HTTP/1.1" 200 None 2026-03-10 01:10:49,069 - lava_callback - INFO - Completed processing callback for node 69af6f87275c8d3b7939a752 2026-03-10 01:10:51,786 - lava_callback - INFO - Received callback for node 69accc97bcd207e915d775b1 INFO: 172.20.0.1:37092 - "POST /node/69accc97bcd207e915d775b1 HTTP/1.0" 202 Accepted 2026-03-10 01:10:51,791 - lava_callback - INFO - Starting async job submit for node 69accc97bcd207e915d775b1 2026-03-10 01:10:51,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:10:51,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69accc97bcd207e915d775b1 HTTP/1.1" 200 None 2026-03-10 01:10:51,809 - lava_callback - DEBUG - Processing node 69accc97bcd207e915d775b1: baseline-arm64-baylibre 2026-03-10 01:10:51,809 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69accc97bcd207e915d775b1 2026-03-10 01:10:51,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:10:51,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69accc97bcd207e915d775b1 HTTP/1.1" 200 None 2026-03-10 01:10:51,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:10:51,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a70fafe31b554803095f11 HTTP/1.1" 200 None 2026-03-10 01:10:51,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:10:51,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69accc97bcd207e915d775b1 HTTP/1.1" 200 None 2026-03-10 01:10:51,879 - lava_callback - ERROR - Error processing callback for node 69accc97bcd207e915d775b1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-10 01:10:52,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:10:52,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:37098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40778 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52106 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:12:13,121 - lava_callback - INFO - Received callback for node 69af6fd2275c8d3b7939a75f INFO: 172.20.0.1:43780 - "POST /node/69af6fd2275c8d3b7939a75f HTTP/1.0" 202 Accepted 2026-03-10 01:12:13,127 - lava_callback - INFO - Starting async job submit for node 69af6fd2275c8d3b7939a75f 2026-03-10 01:12:13,129 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:12:13,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6fd2275c8d3b7939a75f HTTP/1.1" 200 None 2026-03-10 01:12:13,145 - lava_callback - DEBUG - Processing node 69af6fd2275c8d3b7939a75f: baseline-nfs-arm64 2026-03-10 01:12:13,146 - lava_callback - WARNING - Failed to upload log 2026-03-10 01:12:13,147 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:12:13,308 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:12:13,310 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af6fd2275c8d3b7939a75f/lava-logs.yaml 2026-03-10 01:12:13,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:12:13,450 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:12:13,452 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af6fd2275c8d3b7939a75f/lava_callback.json.gz 2026-03-10 01:12:13,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:12:13,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6fd2275c8d3b7939a75f HTTP/1.1" 200 None 2026-03-10 01:12:13,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:12:13,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 01:12:13,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:12:13,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af6fd2275c8d3b7939a75f HTTP/1.1" 200 None 2026-03-10 01:12:13,519 - lava_callback - INFO - Completed processing callback for node 69af6fd2275c8d3b7939a75f INFO: 172.20.0.1:43792 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43796 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:12:22,140 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:12:22,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:35086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56808 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59086 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:13:37,117 - lava_callback - INFO - Received callback for node 69af701a275c8d3b7939a76f INFO: 172.20.0.1:59098 - "POST /node/69af701a275c8d3b7939a76f HTTP/1.0" 202 Accepted 2026-03-10 01:13:37,123 - lava_callback - INFO - Starting async job submit for node 69af701a275c8d3b7939a76f 2026-03-10 01:13:37,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:13:37,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af701a275c8d3b7939a76f HTTP/1.1" 200 None 2026-03-10 01:13:37,157 - lava_callback - DEBUG - Processing node 69af701a275c8d3b7939a76f: baseline-arm64 2026-03-10 01:13:37,209 - lava_callback - WARNING - Failed to upload log 2026-03-10 01:13:37,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:13:37,432 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:13:37,433 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af701a275c8d3b7939a76f/lava-logs.yaml 2026-03-10 01:13:37,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:13:37,580 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:13:37,582 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af701a275c8d3b7939a76f/lava_callback.json.gz 2026-03-10 01:13:37,599 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:13:37,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af701a275c8d3b7939a76f HTTP/1.1" 200 None 2026-03-10 01:13:37,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:13:37,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 01:13:37,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:13:37,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af701a275c8d3b7939a76f HTTP/1.1" 200 None 2026-03-10 01:13:37,684 - lava_callback - INFO - Completed processing callback for node 69af701a275c8d3b7939a76f INFO: 172.20.0.1:40840 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:13:52,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:13:52,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:54374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46714 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46726 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:14:22,519 - lava_callback - INFO - Received callback for node 69af6c43275c8d3b7939a5b9 INFO: 172.20.0.1:54668 - "POST /node/69af6c43275c8d3b7939a5b9 HTTP/1.0" 202 Accepted 2026-03-10 01:14:22,526 - lava_callback - INFO - Starting async job submit for node 69af6c43275c8d3b7939a5b9 2026-03-10 01:14:22,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:14:22,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6c43275c8d3b7939a5b9 HTTP/1.1" 200 None 2026-03-10 01:14:22,598 - lava_callback - DEBUG - Processing node 69af6c43275c8d3b7939a5b9: ltp-smoke 2026-03-10 01:14:23,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:14:23,231 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:14:23,233 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69af6c43275c8d3b7939a5b9/log.txt.gz 2026-03-10 01:14:23,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:14:23,719 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:14:23,722 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69af6c43275c8d3b7939a5b9/lava-logs.yaml 2026-03-10 01:14:23,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:14:23,885 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:14:23,887 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69af6c43275c8d3b7939a5b9/lava_callback.json.gz 2026-03-10 01:14:24,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:14:24,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6c43275c8d3b7939a5b9 HTTP/1.1" 200 None 2026-03-10 01:14:24,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:14:24,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6334275c8d3b79399dba HTTP/1.1" 200 None 2026-03-10 01:14:24,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:14:24,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af6c43275c8d3b7939a5b9 HTTP/1.1" 200 None 2026-03-10 01:14:24,194 - lava_callback - INFO - Completed processing callback for node 69af6c43275c8d3b7939a5b9 INFO: 172.20.0.1:54670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46868 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:14:52,181 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:14:52,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:48100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39026 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:15:08,653 - lava_callback - INFO - Received callback for node 69af7076275c8d3b7939a785 INFO: 172.20.0.1:39038 - "POST /node/69af7076275c8d3b7939a785 HTTP/1.0" 202 Accepted 2026-03-10 01:15:08,659 - lava_callback - INFO - Starting async job submit for node 69af7076275c8d3b7939a785 2026-03-10 01:15:08,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:15:08,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af7076275c8d3b7939a785 HTTP/1.1" 200 None 2026-03-10 01:15:08,677 - lava_callback - DEBUG - Processing node 69af7076275c8d3b7939a785: baseline-arm64-broonie 2026-03-10 01:15:08,682 - lava_callback - WARNING - Failed to upload log 2026-03-10 01:15:08,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:15:08,855 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:15:08,857 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af7076275c8d3b7939a785/lava-logs.yaml 2026-03-10 01:15:08,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:15:09,001 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:15:09,003 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af7076275c8d3b7939a785/lava_callback.json.gz 2026-03-10 01:15:09,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:15:09,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af7076275c8d3b7939a785 HTTP/1.1" 200 None 2026-03-10 01:15:09,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:15:09,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 01:15:09,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:15:09,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af7076275c8d3b7939a785 HTTP/1.1" 200 None 2026-03-10 01:15:09,068 - lava_callback - INFO - Completed processing callback for node 69af7076275c8d3b7939a785 INFO: 172.20.0.1:46486 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46496 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:15:22,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:15:22,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:34958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60732 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:16:09,238 - lava_callback - INFO - Received callback for node 69af70c3275c8d3b7939a797 INFO: 172.20.0.1:60748 - "POST /node/69af70c3275c8d3b7939a797 HTTP/1.0" 202 Accepted 2026-03-10 01:16:09,244 - lava_callback - INFO - Starting async job submit for node 69af70c3275c8d3b7939a797 2026-03-10 01:16:09,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:16:09,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af70c3275c8d3b7939a797 HTTP/1.1" 200 None 2026-03-10 01:16:09,263 - lava_callback - DEBUG - Processing node 69af70c3275c8d3b7939a797: baseline-arm64-clabbe 2026-03-10 01:16:09,267 - lava_callback - WARNING - Failed to upload log 2026-03-10 01:16:09,270 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:16:09,413 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:16:09,415 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69af70c3275c8d3b7939a797/lava-logs.yaml 2026-03-10 01:16:09,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:16:09,559 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:16:09,561 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69af70c3275c8d3b7939a797/lava_callback.json.gz 2026-03-10 01:16:09,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:16:09,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af70c3275c8d3b7939a797 HTTP/1.1" 200 None 2026-03-10 01:16:09,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:16:09,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 01:16:09,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:16:09,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af70c3275c8d3b7939a797 HTTP/1.1" 200 None 2026-03-10 01:16:09,622 - lava_callback - INFO - Completed processing callback for node 69af70c3275c8d3b7939a797 INFO: 172.20.0.1:51782 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51792 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:16:20,836 - lava_callback - INFO - Received callback for node 69af6cee275c8d3b7939a660 INFO: 172.20.0.1:51516 - "POST /node/69af6cee275c8d3b7939a660 HTTP/1.0" 202 Accepted 2026-03-10 01:16:20,848 - lava_callback - INFO - Starting async job submit for node 69af6cee275c8d3b7939a660 2026-03-10 01:16:20,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:16:20,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6cee275c8d3b7939a660 HTTP/1.1" 200 None 2026-03-10 01:16:20,961 - lava_callback - DEBUG - Processing node 69af6cee275c8d3b7939a660: baseline-arm 2026-03-10 01:16:21,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:16:21,864 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:16:21,865 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69af6cee275c8d3b7939a660/log.txt.gz 2026-03-10 01:16:22,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:16:22,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:16:22,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-10 01:16:22,631 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:16:22,634 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69af6cee275c8d3b7939a660/lava-logs.yaml 2026-03-10 01:16:22,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:16:22,853 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:16:22,855 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69af6cee275c8d3b7939a660/lava_callback.json.gz DEBUG: Setup failed for 69af6cee275c8d3b7939a660. Transitting job node result: incomplete -> fail 2026-03-10 01:16:22,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:16:23,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6cee275c8d3b7939a660 HTTP/1.1" 200 None 2026-03-10 01:16:23,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:16:23,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af632c275c8d3b79399db4 HTTP/1.1" 200 None 2026-03-10 01:16:23,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:16:23,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af6cee275c8d3b7939a660 HTTP/1.1" 200 None 2026-03-10 01:16:23,223 - lava_callback - INFO - Completed processing callback for node 69af6cee275c8d3b7939a660 INFO: 172.20.0.1:51522 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:16:27,607 - lava_callback - INFO - Received callback for node 69af6d2c275c8d3b7939a680 INFO: 172.20.0.1:51526 - "POST /node/69af6d2c275c8d3b7939a680 HTTP/1.0" 202 Accepted 2026-03-10 01:16:27,618 - lava_callback - INFO - Starting async job submit for node 69af6d2c275c8d3b7939a680 2026-03-10 01:16:27,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:16:27,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6d2c275c8d3b7939a680 HTTP/1.1" 200 None 2026-03-10 01:16:27,698 - lava_callback - DEBUG - Processing node 69af6d2c275c8d3b7939a680: baseline-x86 2026-03-10 01:16:28,625 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:16:28,842 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:16:28,844 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69af6d2c275c8d3b7939a680/log.txt.gz 2026-03-10 01:16:29,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:16:29,706 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:16:29,709 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69af6d2c275c8d3b7939a680/lava-logs.yaml 2026-03-10 01:16:29,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:16:29,917 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:16:29,918 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69af6d2c275c8d3b7939a680/lava_callback.json.gz DEBUG: Setup failed for 69af6d2c275c8d3b7939a680. Transitting job node result: incomplete -> fail 2026-03-10 01:16:30,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:16:30,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6d2c275c8d3b7939a680 HTTP/1.1" 200 None 2026-03-10 01:16:30,029 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:16:30,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af635a275c8d3b79399ddd HTTP/1.1" 200 None 2026-03-10 01:16:30,045 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:16:30,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af6d2c275c8d3b7939a680 HTTP/1.1" 200 None 2026-03-10 01:16:30,187 - lava_callback - INFO - Completed processing callback for node 69af6d2c275c8d3b7939a680 INFO: 172.20.0.1:33960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33908 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:16:52,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:16:52,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:50898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37648 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33212 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:18:12,094 - lava_callback - INFO - Received callback for node 69acce4fbcd207e915d776a2 INFO: 172.20.0.1:58008 - "POST /node/69acce4fbcd207e915d776a2 HTTP/1.0" 202 Accepted 2026-03-10 01:18:12,100 - lava_callback - INFO - Starting async job submit for node 69acce4fbcd207e915d776a2 2026-03-10 01:18:12,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:18:12,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acce4fbcd207e915d776a2 HTTP/1.1" 200 None 2026-03-10 01:18:12,118 - lava_callback - DEBUG - Processing node 69acce4fbcd207e915d776a2: baseline-arm64-baylibre 2026-03-10 01:18:12,118 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acce4fbcd207e915d776a2 2026-03-10 01:18:12,119 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:18:12,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acce4fbcd207e915d776a2 HTTP/1.1" 200 None 2026-03-10 01:18:12,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:18:12,149 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a70fafe31b554803095f11 HTTP/1.1" 200 None 2026-03-10 01:18:12,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:18:12,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acce4fbcd207e915d776a2 HTTP/1.1" 200 None 2026-03-10 01:18:12,179 - lava_callback - ERROR - Error processing callback for node 69acce4fbcd207e915d776a2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58014 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38836 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:18:29,346 - lava_callback - INFO - Received callback for node 69af7101275c8d3b7939a7b1 INFO: 172.20.0.1:38842 - "POST /node/69af7101275c8d3b7939a7b1 HTTP/1.0" 202 Accepted 2026-03-10 01:18:29,356 - lava_callback - INFO - Starting async job submit for node 69af7101275c8d3b7939a7b1 2026-03-10 01:18:29,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:18:29,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af7101275c8d3b7939a7b1 HTTP/1.1" 200 None 2026-03-10 01:18:29,416 - lava_callback - DEBUG - Processing node 69af7101275c8d3b7939a7b1: baseline-arm64 2026-03-10 01:18:29,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:18:29,958 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:18:29,959 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af7101275c8d3b7939a7b1/log.txt.gz 2026-03-10 01:18:30,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:18:30,474 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:18:30,477 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af7101275c8d3b7939a7b1/lava-logs.yaml 2026-03-10 01:18:30,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:18:30,661 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:18:30,663 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af7101275c8d3b7939a7b1/lava_callback.json.gz DEBUG: Setup failed for 69af7101275c8d3b7939a7b1. Transitting job node result: incomplete -> fail 2026-03-10 01:18:30,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:18:30,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af7101275c8d3b7939a7b1 HTTP/1.1" 200 None 2026-03-10 01:18:30,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:18:30,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 01:18:30,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:18:30,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af7101275c8d3b7939a7b1 HTTP/1.1" 200 None 2026-03-10 01:18:30,959 - lava_callback - INFO - Completed processing callback for node 69af7101275c8d3b7939a7b1 INFO: 172.20.0.1:41752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33092 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:18:52,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:18:52,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:33770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46212 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46226 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:19:21,092 - lava_callback - INFO - Received callback for node 69af7162275c8d3b7939a7c5 INFO: 172.20.0.1:49240 - "POST /node/69af7162275c8d3b7939a7c5 HTTP/1.0" 202 Accepted 2026-03-10 01:19:21,098 - lava_callback - INFO - Starting async job submit for node 69af7162275c8d3b7939a7c5 2026-03-10 01:19:21,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:19:21,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af7162275c8d3b7939a7c5 HTTP/1.1" 200 None 2026-03-10 01:19:21,117 - lava_callback - DEBUG - Processing node 69af7162275c8d3b7939a7c5: baseline-arm64-broonie 2026-03-10 01:19:21,123 - lava_callback - WARNING - Failed to upload log 2026-03-10 01:19:21,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:19:21,249 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:19:21,251 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af7162275c8d3b7939a7c5/lava-logs.yaml 2026-03-10 01:19:21,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:19:21,379 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:19:21,381 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af7162275c8d3b7939a7c5/lava_callback.json.gz 2026-03-10 01:19:21,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:19:21,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af7162275c8d3b7939a7c5 HTTP/1.1" 200 None 2026-03-10 01:19:21,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:19:21,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 01:19:21,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:19:21,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af7162275c8d3b7939a7c5 HTTP/1.1" 200 None 2026-03-10 01:19:21,456 - lava_callback - INFO - Completed processing callback for node 69af7162275c8d3b7939a7c5 2026-03-10 01:19:22,351 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:19:22,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:49256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39226 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:19:40,850 - lava_callback - INFO - Received callback for node 69af6c02275c8d3b7939a52c INFO: 172.20.0.1:58776 - "POST /node/69af6c02275c8d3b7939a52c HTTP/1.0" 202 Accepted 2026-03-10 01:19:40,856 - lava_callback - INFO - Starting async job submit for node 69af6c02275c8d3b7939a52c 2026-03-10 01:19:40,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:19:40,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6c02275c8d3b7939a52c HTTP/1.1" 200 None 2026-03-10 01:19:40,889 - lava_callback - DEBUG - Processing node 69af6c02275c8d3b7939a52c: baseline-arm64-broonie 2026-03-10 01:19:41,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:19:41,328 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:19:41,330 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af6c02275c8d3b7939a52c/log.txt.gz 2026-03-10 01:19:41,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:19:41,747 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:19:41,749 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af6c02275c8d3b7939a52c/lava-logs.yaml 2026-03-10 01:19:41,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:19:41,887 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:19:41,889 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af6c02275c8d3b7939a52c/lava_callback.json.gz 2026-03-10 01:19:41,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:19:41,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6c02275c8d3b7939a52c HTTP/1.1" 200 None 2026-03-10 01:19:41,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:19:41,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 01:19:41,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:19:41,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af6c02275c8d3b7939a52c HTTP/1.1" 200 None 2026-03-10 01:19:42,047 - lava_callback - INFO - Completed processing callback for node 69af6c02275c8d3b7939a52c INFO: 172.20.0.1:58784 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:19:52,178 - lava_callback - INFO - Received callback for node 69acceadbcd207e915d7774b INFO: 172.20.0.1:40408 - "POST /node/69acceadbcd207e915d7774b HTTP/1.0" 202 Accepted 2026-03-10 01:19:52,184 - lava_callback - INFO - Starting async job submit for node 69acceadbcd207e915d7774b 2026-03-10 01:19:52,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:19:52,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acceadbcd207e915d7774b HTTP/1.1" 200 None 2026-03-10 01:19:52,200 - lava_callback - DEBUG - Processing node 69acceadbcd207e915d7774b: baseline-arm64-baylibre 2026-03-10 01:19:52,200 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acceadbcd207e915d7774b 2026-03-10 01:19:52,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:19:52,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acceadbcd207e915d7774b HTTP/1.1" 200 None 2026-03-10 01:19:52,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:19:52,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acc01dbcd207e915d76ba7 HTTP/1.1" 200 None 2026-03-10 01:19:52,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:19:52,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acceadbcd207e915d7774b HTTP/1.1" 200 None 2026-03-10 01:19:52,259 - lava_callback - ERROR - Error processing callback for node 69acceadbcd207e915d7774b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-10 01:19:52,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:19:52,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-10 01:19:52,849 - lava_callback - INFO - Received callback for node 69acceaebcd207e915d77773 INFO: 172.20.0.1:40422 - "POST /node/69acceaebcd207e915d77773 HTTP/1.0" 202 Accepted 2026-03-10 01:19:52,855 - lava_callback - INFO - Starting async job submit for node 69acceaebcd207e915d77773 2026-03-10 01:19:52,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:19:52,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acceaebcd207e915d77773 HTTP/1.1" 200 None 2026-03-10 01:19:52,870 - lava_callback - DEBUG - Processing node 69acceaebcd207e915d77773: baseline-arm64-baylibre 2026-03-10 01:19:52,871 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acceaebcd207e915d77773 2026-03-10 01:19:52,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:19:52,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acceaebcd207e915d77773 HTTP/1.1" 200 None 2026-03-10 01:19:52,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:19:52,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acc01dbcd207e915d76ba7 HTTP/1.1" 200 None 2026-03-10 01:19:52,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:19:52,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acceaebcd207e915d77773 HTTP/1.1" 200 None 2026-03-10 01:19:52,926 - lava_callback - ERROR - Error processing callback for node 69acceaebcd207e915d77773: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-10 01:19:53,518 - lava_callback - INFO - Received callback for node 69acceb0bcd207e915d7777d INFO: 172.20.0.1:40436 - "POST /node/69acceb0bcd207e915d7777d HTTP/1.0" 202 Accepted 2026-03-10 01:19:53,523 - lava_callback - INFO - Starting async job submit for node 69acceb0bcd207e915d7777d 2026-03-10 01:19:53,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:19:53,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acceb0bcd207e915d7777d HTTP/1.1" 200 None 2026-03-10 01:19:53,540 - lava_callback - DEBUG - Processing node 69acceb0bcd207e915d7777d: baseline-arm64-baylibre 2026-03-10 01:19:53,540 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acceb0bcd207e915d7777d 2026-03-10 01:19:53,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:19:53,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acceb0bcd207e915d7777d HTTP/1.1" 200 None 2026-03-10 01:19:53,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:19:53,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acc01dbcd207e915d76ba7 HTTP/1.1" 200 None 2026-03-10 01:19:53,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:19:53,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acceb0bcd207e915d7777d HTTP/1.1" 200 None 2026-03-10 01:19:53,593 - lava_callback - ERROR - Error processing callback for node 69acceb0bcd207e915d7777d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-10 01:19:54,187 - lava_callback - INFO - Received callback for node 69acceb2bcd207e915d77780 INFO: 172.20.0.1:40448 - "POST /node/69acceb2bcd207e915d77780 HTTP/1.0" 202 Accepted 2026-03-10 01:19:54,193 - lava_callback - INFO - Starting async job submit for node 69acceb2bcd207e915d77780 2026-03-10 01:19:54,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:19:54,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acceb2bcd207e915d77780 HTTP/1.1" 200 None 2026-03-10 01:19:54,210 - lava_callback - DEBUG - Processing node 69acceb2bcd207e915d77780: baseline-arm64-baylibre 2026-03-10 01:19:54,210 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acceb2bcd207e915d77780 2026-03-10 01:19:54,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:19:54,225 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acceb2bcd207e915d77780 HTTP/1.1" 200 None 2026-03-10 01:19:54,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:19:54,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acc01dbcd207e915d76ba7 HTTP/1.1" 200 None 2026-03-10 01:19:54,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:19:54,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acceb2bcd207e915d77780 HTTP/1.1" 200 None 2026-03-10 01:19:54,271 - lava_callback - ERROR - Error processing callback for node 69acceb2bcd207e915d77780: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44258 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44274 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:20:20,224 - lava_callback - INFO - Received callback for node 69accec3bcd207e915d777a4 INFO: 172.20.0.1:43366 - "POST /node/69accec3bcd207e915d777a4 HTTP/1.0" 202 Accepted 2026-03-10 01:20:20,230 - lava_callback - INFO - Starting async job submit for node 69accec3bcd207e915d777a4 2026-03-10 01:20:20,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:20:20,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69accec3bcd207e915d777a4 HTTP/1.1" 200 None 2026-03-10 01:20:20,256 - lava_callback - DEBUG - Processing node 69accec3bcd207e915d777a4: baseline-arm64-clabbe 2026-03-10 01:20:20,257 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69accec3bcd207e915d777a4 2026-03-10 01:20:20,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:20:20,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69accec3bcd207e915d777a4 HTTP/1.1" 200 None 2026-03-10 01:20:20,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:20:20,286 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acc01dbcd207e915d76ba7 HTTP/1.1" 200 None 2026-03-10 01:20:20,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:20:20,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69accec3bcd207e915d777a4 HTTP/1.1" 200 None 2026-03-10 01:20:20,317 - lava_callback - ERROR - Error processing callback for node 69accec3bcd207e915d777a4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-10 01:20:20,880 - lava_callback - INFO - Received callback for node 69accecabcd207e915d777bc INFO: 172.20.0.1:43380 - "POST /node/69accecabcd207e915d777bc HTTP/1.0" 202 Accepted 2026-03-10 01:20:20,885 - lava_callback - INFO - Starting async job submit for node 69accecabcd207e915d777bc 2026-03-10 01:20:20,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:20:20,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69accecabcd207e915d777bc HTTP/1.1" 200 None 2026-03-10 01:20:20,904 - lava_callback - DEBUG - Processing node 69accecabcd207e915d777bc: baseline-arm64-clabbe 2026-03-10 01:20:20,904 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69accecabcd207e915d777bc 2026-03-10 01:20:20,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:20:20,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69accecabcd207e915d777bc HTTP/1.1" 200 None 2026-03-10 01:20:20,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:20:20,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acc01dbcd207e915d76ba7 HTTP/1.1" 200 None 2026-03-10 01:20:20,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:20:20,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69accecabcd207e915d777bc HTTP/1.1" 200 None 2026-03-10 01:20:20,969 - lava_callback - ERROR - Error processing callback for node 69accecabcd207e915d777bc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43056 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:21:02,406 - lava_callback - INFO - Received callback for node 69af719f275c8d3b7939a7d6 INFO: 172.20.0.1:41002 - "POST /node/69af719f275c8d3b7939a7d6 HTTP/1.0" 202 Accepted 2026-03-10 01:21:02,412 - lava_callback - INFO - Starting async job submit for node 69af719f275c8d3b7939a7d6 2026-03-10 01:21:02,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:21:02,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af719f275c8d3b7939a7d6 HTTP/1.1" 200 None 2026-03-10 01:21:02,462 - lava_callback - DEBUG - Processing node 69af719f275c8d3b7939a7d6: baseline-arm64 2026-03-10 01:21:02,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:21:03,037 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:21:03,038 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af719f275c8d3b7939a7d6/log.txt.gz 2026-03-10 01:21:03,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:21:03,492 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:21:03,494 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af719f275c8d3b7939a7d6/lava-logs.yaml 2026-03-10 01:21:03,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:41006 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:21:03,659 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:21:03,661 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af719f275c8d3b7939a7d6/lava_callback.json.gz DEBUG: Setup failed for 69af719f275c8d3b7939a7d6. Transitting job node result: incomplete -> fail 2026-03-10 01:21:03,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:21:03,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af719f275c8d3b7939a7d6 HTTP/1.1" 200 None 2026-03-10 01:21:03,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:21:03,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 01:21:03,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:21:03,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af719f275c8d3b7939a7d6 HTTP/1.1" 200 None 2026-03-10 01:21:03,900 - lava_callback - INFO - Completed processing callback for node 69af719f275c8d3b7939a7d6 INFO: 172.20.0.1:41014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38242 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38254 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:21:22,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:21:22,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:59780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46274 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38282 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:22:30,346 - lava_callback - INFO - Received callback for node 69af71f4275c8d3b7939a7f5 INFO: 172.20.0.1:52770 - "POST /node/69af71f4275c8d3b7939a7f5 HTTP/1.0" 202 Accepted 2026-03-10 01:22:30,353 - lava_callback - INFO - Starting async job submit for node 69af71f4275c8d3b7939a7f5 2026-03-10 01:22:30,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:22:30,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af71f4275c8d3b7939a7f5 HTTP/1.1" 200 None 2026-03-10 01:22:30,412 - lava_callback - DEBUG - Processing node 69af71f4275c8d3b7939a7f5: baseline-arm64 2026-03-10 01:22:30,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:22:30,941 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:22:30,943 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af71f4275c8d3b7939a7f5/log.txt.gz 2026-03-10 01:22:31,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:22:31,396 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:22:31,399 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af71f4275c8d3b7939a7f5/lava-logs.yaml 2026-03-10 01:22:31,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:22:31,556 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:22:31,558 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af71f4275c8d3b7939a7f5/lava_callback.json.gz DEBUG: Setup failed for 69af71f4275c8d3b7939a7f5. Transitting job node result: incomplete -> fail 2026-03-10 01:22:31,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:22:31,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af71f4275c8d3b7939a7f5 HTTP/1.1" 200 None 2026-03-10 01:22:31,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:22:31,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 01:22:31,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:22:31,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af71f4275c8d3b7939a7f5 HTTP/1.1" 200 None 2026-03-10 01:22:31,756 - lava_callback - INFO - Completed processing callback for node 69af71f4275c8d3b7939a7f5 INFO: 172.20.0.1:52784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48820 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:22:52,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:22:52,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:49040 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:22:57,191 - lava_callback - INFO - Received callback for node 69af6d2e275c8d3b7939a686 INFO: 172.20.0.1:49054 - "POST /node/69af6d2e275c8d3b7939a686 HTTP/1.0" 202 Accepted 2026-03-10 01:22:57,197 - lava_callback - INFO - Starting async job submit for node 69af6d2e275c8d3b7939a686 2026-03-10 01:22:57,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:22:57,236 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6d2e275c8d3b7939a686 HTTP/1.1" 200 None 2026-03-10 01:22:57,237 - lava_callback - DEBUG - Processing node 69af6d2e275c8d3b7939a686: baseline-x86 2026-03-10 01:22:57,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:22:57,680 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:22:57,682 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69af6d2e275c8d3b7939a686/log.txt.gz 2026-03-10 01:22:57,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:22:58,035 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:22:58,038 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69af6d2e275c8d3b7939a686/lava-logs.yaml 2026-03-10 01:22:58,045 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:22:58,207 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:22:58,209 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69af6d2e275c8d3b7939a686/lava_callback.json.gz 2026-03-10 01:22:58,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:22:58,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6d2e275c8d3b7939a686 HTTP/1.1" 200 None 2026-03-10 01:22:58,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:22:58,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af635a275c8d3b79399ddd HTTP/1.1" 200 None 2026-03-10 01:22:58,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:22:58,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af6d2e275c8d3b7939a686 HTTP/1.1" 200 None 2026-03-10 01:22:58,296 - lava_callback - INFO - Completed processing callback for node 69af6d2e275c8d3b7939a686 INFO: 172.20.0.1:58558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47060 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47066 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:23:22,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:23:22,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:59034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48076 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43128 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53784 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60956 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60394 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37390 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51382 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39582 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51252 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:31:32,810 - lava_callback - INFO - Received callback for node 69acd165bcd207e915d779f9 INFO: 172.20.0.1:51162 - "POST /node/69acd165bcd207e915d779f9 HTTP/1.0" 202 Accepted 2026-03-10 01:31:32,816 - lava_callback - INFO - Starting async job submit for node 69acd165bcd207e915d779f9 2026-03-10 01:31:32,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:31:32,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acd165bcd207e915d779f9 HTTP/1.1" 200 None 2026-03-10 01:31:32,831 - lava_callback - DEBUG - Processing node 69acd165bcd207e915d779f9: baseline-arm64-baylibre 2026-03-10 01:31:32,832 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acd165bcd207e915d779f9 2026-03-10 01:31:32,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:31:32,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acd165bcd207e915d779f9 HTTP/1.1" 200 None 2026-03-10 01:31:32,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:31:32,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a77a6ba255cc0324eb011a HTTP/1.1" 200 None 2026-03-10 01:31:32,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:31:32,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acd165bcd207e915d779f9 HTTP/1.1" 200 None 2026-03-10 01:31:32,902 - lava_callback - ERROR - Error processing callback for node 69acd165bcd207e915d779f9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51174 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:31:40,422 - lava_callback - INFO - Received callback for node 69af729c275c8d3b7939a816 INFO: 172.20.0.1:32864 - "POST /node/69af729c275c8d3b7939a816 HTTP/1.0" 202 Accepted 2026-03-10 01:31:40,429 - lava_callback - INFO - Starting async job submit for node 69af729c275c8d3b7939a816 2026-03-10 01:31:40,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:31:40,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af729c275c8d3b7939a816 HTTP/1.1" 200 None 2026-03-10 01:31:40,477 - lava_callback - DEBUG - Processing node 69af729c275c8d3b7939a816: baseline-nfs-arm64 2026-03-10 01:31:40,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:31:41,048 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:31:41,050 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af729c275c8d3b7939a816/log.txt.gz 2026-03-10 01:31:41,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:31:41,502 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:31:41,505 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af729c275c8d3b7939a816/lava-logs.yaml 2026-03-10 01:31:41,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:31:41,682 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:31:41,684 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af729c275c8d3b7939a816/lava_callback.json.gz DEBUG: Setup failed for 69af729c275c8d3b7939a816. Transitting job node result: incomplete -> fail 2026-03-10 01:31:41,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:31:41,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af729c275c8d3b7939a816 HTTP/1.1" 200 None 2026-03-10 01:31:41,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:31:41,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 01:31:41,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:31:41,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af729c275c8d3b7939a816 HTTP/1.1" 200 None 2026-03-10 01:31:41,881 - lava_callback - INFO - Completed processing callback for node 69af729c275c8d3b7939a816 INFO: 172.20.0.1:32876 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:31:52,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:31:52,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:39486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35696 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52422 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:32:52,871 - lava_callback - INFO - Received callback for node 69acd1b7bcd207e915d77a08 INFO: 172.20.0.1:47984 - "POST /node/69acd1b7bcd207e915d77a08 HTTP/1.0" 202 Accepted 2026-03-10 01:32:52,876 - lava_callback - INFO - Starting async job submit for node 69acd1b7bcd207e915d77a08 2026-03-10 01:32:52,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:32:52,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acd1b7bcd207e915d77a08 HTTP/1.1" 200 None 2026-03-10 01:32:52,909 - lava_callback - DEBUG - Processing node 69acd1b7bcd207e915d77a08: baseline-arm64-baylibre 2026-03-10 01:32:52,909 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acd1b7bcd207e915d77a08 2026-03-10 01:32:52,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:32:52,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acd1b7bcd207e915d77a08 HTTP/1.1" 200 None 2026-03-10 01:32:52,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:32:52,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a77a6ba255cc0324eb011a HTTP/1.1" 200 None 2026-03-10 01:32:52,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:32:52,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acd1b7bcd207e915d77a08 HTTP/1.1" 200 None 2026-03-10 01:32:52,972 - lava_callback - ERROR - Error processing callback for node 69acd1b7bcd207e915d77a08: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50838 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45374 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:33:40,624 - lava_callback - INFO - Received callback for node 69af748d275c8d3b7939a856 INFO: 172.20.0.1:36596 - "POST /node/69af748d275c8d3b7939a856 HTTP/1.0" 202 Accepted 2026-03-10 01:33:40,631 - lava_callback - INFO - Starting async job submit for node 69af748d275c8d3b7939a856 2026-03-10 01:33:40,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:33:40,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af748d275c8d3b7939a856 HTTP/1.1" 200 None 2026-03-10 01:33:40,682 - lava_callback - DEBUG - Processing node 69af748d275c8d3b7939a856: baseline-nfs-arm64 2026-03-10 01:33:41,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:33:41,205 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:33:41,207 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af748d275c8d3b7939a856/log.txt.gz 2026-03-10 01:33:41,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:33:41,655 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:33:41,657 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af748d275c8d3b7939a856/lava-logs.yaml 2026-03-10 01:33:41,665 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:33:41,816 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:33:41,818 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af748d275c8d3b7939a856/lava_callback.json.gz DEBUG: Setup failed for 69af748d275c8d3b7939a856. Transitting job node result: incomplete -> fail 2026-03-10 01:33:41,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:33:41,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af748d275c8d3b7939a856 HTTP/1.1" 200 None 2026-03-10 01:33:41,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:33:41,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 01:33:41,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:33:41,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af748d275c8d3b7939a856 HTTP/1.1" 200 None 2026-03-10 01:33:42,019 - lava_callback - INFO - Completed processing callback for node 69af748d275c8d3b7939a856 INFO: 172.20.0.1:36600 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:33:52,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:33:52,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:37968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35256 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33420 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49296 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:35:32,958 - lava_callback - INFO - Received callback for node 69acd256bcd207e915d77a5f INFO: 172.20.0.1:55280 - "POST /node/69acd256bcd207e915d77a5f HTTP/1.0" 202 Accepted 2026-03-10 01:35:32,968 - lava_callback - INFO - Starting async job submit for node 69acd256bcd207e915d77a5f 2026-03-10 01:35:32,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:35:32,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acd256bcd207e915d77a5f HTTP/1.1" 200 None 2026-03-10 01:35:33,000 - lava_callback - DEBUG - Processing node 69acd256bcd207e915d77a5f: baseline-arm64-baylibre 2026-03-10 01:35:33,000 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acd256bcd207e915d77a5f 2026-03-10 01:35:33,001 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:35:33,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acd256bcd207e915d77a5f HTTP/1.1" 200 None 2026-03-10 01:35:33,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:35:33,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a77a6ba255cc0324eb011a HTTP/1.1" 200 None 2026-03-10 01:35:33,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:35:33,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acd256bcd207e915d77a5f HTTP/1.1" 200 None 2026-03-10 01:35:33,100 - lava_callback - ERROR - Error processing callback for node 69acd256bcd207e915d77a5f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43402 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43414 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:36:17,312 - lava_callback - INFO - Received callback for node 69af753e275c8d3b7939a870 INFO: 172.20.0.1:43430 - "POST /node/69af753e275c8d3b7939a870 HTTP/1.0" 202 Accepted 2026-03-10 01:36:17,322 - lava_callback - INFO - Starting async job submit for node 69af753e275c8d3b7939a870 2026-03-10 01:36:17,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:36:17,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af753e275c8d3b7939a870 HTTP/1.1" 200 None 2026-03-10 01:36:17,422 - lava_callback - DEBUG - Processing node 69af753e275c8d3b7939a870: baseline-arm64-foundriesio 2026-03-10 01:36:17,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:36:17,919 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:36:17,921 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69af753e275c8d3b7939a870/log.txt.gz 2026-03-10 01:36:18,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:36:18,526 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:36:18,527 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69af753e275c8d3b7939a870/lava-logs.yaml 2026-03-10 01:36:18,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:36:18,720 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:36:18,722 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69af753e275c8d3b7939a870/lava_callback.json.gz DEBUG: Setup failed for 69af753e275c8d3b7939a870. Transitting job node result: incomplete -> fail 2026-03-10 01:36:18,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:36:18,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af753e275c8d3b7939a870 HTTP/1.1" 200 None 2026-03-10 01:36:18,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:36:18,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 01:36:18,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:36:18,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af753e275c8d3b7939a870 HTTP/1.1" 200 None 2026-03-10 01:36:19,069 - lava_callback - INFO - Completed processing callback for node 69af753e275c8d3b7939a870 2026-03-10 01:36:22,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:36:22,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:55352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48342 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:36:53,225 - lava_callback - INFO - Received callback for node 69acd2aabcd207e915d77a73 INFO: 172.20.0.1:60268 - "POST /node/69acd2aabcd207e915d77a73 HTTP/1.0" 202 Accepted 2026-03-10 01:36:53,231 - lava_callback - INFO - Starting async job submit for node 69acd2aabcd207e915d77a73 2026-03-10 01:36:53,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:36:53,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acd2aabcd207e915d77a73 HTTP/1.1" 200 None 2026-03-10 01:36:53,251 - lava_callback - DEBUG - Processing node 69acd2aabcd207e915d77a73: baseline-arm64-baylibre 2026-03-10 01:36:53,251 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acd2aabcd207e915d77a73 2026-03-10 01:36:53,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:36:53,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acd2aabcd207e915d77a73 HTTP/1.1" 200 None 2026-03-10 01:36:53,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:36:53,287 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a77a6ba255cc0324eb011a HTTP/1.1" 200 None 2026-03-10 01:36:53,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:36:53,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acd2aabcd207e915d77a73 HTTP/1.1" 200 None 2026-03-10 01:36:53,315 - lava_callback - ERROR - Error processing callback for node 69acd2aabcd207e915d77a73: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36756 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38780 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38496 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34896 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:39:33,216 - lava_callback - INFO - Received callback for node 69acd352bcd207e915d77aca INFO: 172.20.0.1:51818 - "POST /node/69acd352bcd207e915d77aca HTTP/1.0" 202 Accepted 2026-03-10 01:39:33,222 - lava_callback - INFO - Starting async job submit for node 69acd352bcd207e915d77aca 2026-03-10 01:39:33,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:39:33,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acd352bcd207e915d77aca HTTP/1.1" 200 None 2026-03-10 01:39:33,238 - lava_callback - DEBUG - Processing node 69acd352bcd207e915d77aca: baseline-arm64-baylibre 2026-03-10 01:39:33,238 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acd352bcd207e915d77aca 2026-03-10 01:39:33,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:39:33,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acd352bcd207e915d77aca HTTP/1.1" 200 None 2026-03-10 01:39:33,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:39:33,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a77a6ba255cc0324eb011a HTTP/1.1" 200 None 2026-03-10 01:39:33,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:39:33,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acd352bcd207e915d77aca HTTP/1.1" 200 None 2026-03-10 01:39:33,290 - lava_callback - ERROR - Error processing callback for node 69acd352bcd207e915d77aca: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40760 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52556 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52568 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:41:23,908 - lava_callback - INFO - Received callback for node 69af76bf275c8d3b7939a8a5 INFO: 172.20.0.1:60972 - "POST /node/69af76bf275c8d3b7939a8a5 HTTP/1.0" 202 Accepted 2026-03-10 01:41:23,914 - lava_callback - INFO - Starting async job submit for node 69af76bf275c8d3b7939a8a5 2026-03-10 01:41:23,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:41:23,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af76bf275c8d3b7939a8a5 HTTP/1.1" 200 None 2026-03-10 01:41:23,931 - lava_callback - DEBUG - Processing node 69af76bf275c8d3b7939a8a5: baseline-arm64 2026-03-10 01:41:23,935 - lava_callback - WARNING - Failed to upload log 2026-03-10 01:41:23,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:41:24,102 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:41:24,104 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af76bf275c8d3b7939a8a5/lava-logs.yaml 2026-03-10 01:41:24,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:41:24,268 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:41:24,270 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af76bf275c8d3b7939a8a5/lava_callback.json.gz 2026-03-10 01:41:24,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:41:24,288 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af76bf275c8d3b7939a8a5 HTTP/1.1" 200 None 2026-03-10 01:41:24,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:41:24,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 01:41:24,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:41:24,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af76bf275c8d3b7939a8a5 HTTP/1.1" 200 None 2026-03-10 01:41:24,339 - lava_callback - INFO - Completed processing callback for node 69af76bf275c8d3b7939a8a5 INFO: 172.20.0.1:60982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50354 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:41:52,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:41:52,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:46710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50816 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40516 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36994 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:43:33,461 - lava_callback - INFO - Received callback for node 69acd443bcd207e915d77b2e INFO: 172.20.0.1:51136 - "POST /node/69acd443bcd207e915d77b2e HTTP/1.0" 202 Accepted 2026-03-10 01:43:33,466 - lava_callback - INFO - Starting async job submit for node 69acd443bcd207e915d77b2e 2026-03-10 01:43:33,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:43:33,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acd443bcd207e915d77b2e HTTP/1.1" 200 None 2026-03-10 01:43:33,483 - lava_callback - DEBUG - Processing node 69acd443bcd207e915d77b2e: baseline-arm64-baylibre 2026-03-10 01:43:33,483 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acd443bcd207e915d77b2e 2026-03-10 01:43:33,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:43:33,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acd443bcd207e915d77b2e HTTP/1.1" 200 None 2026-03-10 01:43:33,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:43:33,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a77a6ba255cc0324eb011a HTTP/1.1" 200 None 2026-03-10 01:43:33,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:43:33,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acd443bcd207e915d77b2e HTTP/1.1" 200 None 2026-03-10 01:43:33,535 - lava_callback - ERROR - Error processing callback for node 69acd443bcd207e915d77b2e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58118 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38668 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:44:29,884 - lava_callback - INFO - Received callback for node 69af73e1275c8d3b7939a838 INFO: 172.20.0.1:51514 - "POST /node/69af73e1275c8d3b7939a838 HTTP/1.0" 202 Accepted 2026-03-10 01:44:29,890 - lava_callback - INFO - Starting async job submit for node 69af73e1275c8d3b7939a838 2026-03-10 01:44:29,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:44:29,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af73e1275c8d3b7939a838 HTTP/1.1" 200 None 2026-03-10 01:44:29,936 - lava_callback - DEBUG - Processing node 69af73e1275c8d3b7939a838: baseline-arm64 2026-03-10 01:44:30,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:44:30,615 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:44:30,617 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af73e1275c8d3b7939a838/log.txt.gz 2026-03-10 01:44:30,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:44:31,163 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:44:31,165 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af73e1275c8d3b7939a838/lava-logs.yaml 2026-03-10 01:44:31,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:44:31,360 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:44:31,362 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af73e1275c8d3b7939a838/lava_callback.json.gz DEBUG: Setup failed for 69af73e1275c8d3b7939a838. Transitting job node result: incomplete -> fail 2026-03-10 01:44:31,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:44:31,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af73e1275c8d3b7939a838 HTTP/1.1" 200 None 2026-03-10 01:44:31,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:44:31,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 01:44:31,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:44:31,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af73e1275c8d3b7939a838 HTTP/1.1" 200 None 2026-03-10 01:44:31,580 - lava_callback - INFO - Completed processing callback for node 69af73e1275c8d3b7939a838 INFO: 172.20.0.1:51522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38378 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:44:52,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:44:52,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:32946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34968 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35802 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45842 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57290 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:47:33,690 - lava_callback - INFO - Received callback for node 69acd527bcd207e915d77b8d INFO: 172.20.0.1:50640 - "POST /node/69acd527bcd207e915d77b8d HTTP/1.0" 202 Accepted 2026-03-10 01:47:33,696 - lava_callback - INFO - Starting async job submit for node 69acd527bcd207e915d77b8d 2026-03-10 01:47:33,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:47:33,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acd527bcd207e915d77b8d HTTP/1.1" 200 None 2026-03-10 01:47:33,712 - lava_callback - DEBUG - Processing node 69acd527bcd207e915d77b8d: baseline-arm64-baylibre 2026-03-10 01:47:33,713 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acd527bcd207e915d77b8d 2026-03-10 01:47:33,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:47:33,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acd527bcd207e915d77b8d HTTP/1.1" 200 None 2026-03-10 01:47:33,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:47:33,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a70fafe31b554803095f11 HTTP/1.1" 200 None 2026-03-10 01:47:33,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:47:33,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acd527bcd207e915d77b8d HTTP/1.1" 200 None 2026-03-10 01:47:33,775 - lava_callback - ERROR - Error processing callback for node 69acd527bcd207e915d77b8d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57668 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:48:01,413 - lava_callback - INFO - Received callback for node 69af7342275c8d3b7939a82a INFO: 172.20.0.1:42608 - "POST /node/69af7342275c8d3b7939a82a HTTP/1.0" 202 Accepted 2026-03-10 01:48:01,420 - lava_callback - INFO - Starting async job submit for node 69af7342275c8d3b7939a82a 2026-03-10 01:48:01,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:48:01,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af7342275c8d3b7939a82a HTTP/1.1" 200 None 2026-03-10 01:48:01,468 - lava_callback - DEBUG - Processing node 69af7342275c8d3b7939a82a: baseline-nfs-arm64 2026-03-10 01:48:01,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:48:01,929 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:48:01,930 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af7342275c8d3b7939a82a/log.txt.gz 2026-03-10 01:48:02,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:48:02,337 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:48:02,339 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af7342275c8d3b7939a82a/lava-logs.yaml 2026-03-10 01:48:02,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:48:02,499 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:48:02,501 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af7342275c8d3b7939a82a/lava_callback.json.gz DEBUG: Setup failed for 69af7342275c8d3b7939a82a. Transitting job node result: incomplete -> fail 2026-03-10 01:48:02,567 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:48:02,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af7342275c8d3b7939a82a HTTP/1.1" 200 None 2026-03-10 01:48:02,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:48:02,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 01:48:02,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:48:02,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af7342275c8d3b7939a82a HTTP/1.1" 200 None 2026-03-10 01:48:02,701 - lava_callback - INFO - Completed processing callback for node 69af7342275c8d3b7939a82a INFO: 172.20.0.1:42624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45948 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45952 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:48:22,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:48:22,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:40046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53366 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:49:11,592 - lava_callback - INFO - Received callback for node 69af74dc275c8d3b7939a85e INFO: 172.20.0.1:44522 - "POST /node/69af74dc275c8d3b7939a85e HTTP/1.0" 202 Accepted 2026-03-10 01:49:11,599 - lava_callback - INFO - Starting async job submit for node 69af74dc275c8d3b7939a85e 2026-03-10 01:49:11,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:49:11,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af74dc275c8d3b7939a85e HTTP/1.1" 200 None 2026-03-10 01:49:11,646 - lava_callback - DEBUG - Processing node 69af74dc275c8d3b7939a85e: baseline-nfs-arm64 2026-03-10 01:49:12,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:49:12,405 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:49:12,407 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af74dc275c8d3b7939a85e/log.txt.gz 2026-03-10 01:49:12,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:49:12,944 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:49:12,947 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af74dc275c8d3b7939a85e/lava-logs.yaml 2026-03-10 01:49:12,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:49:13,124 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:49:13,126 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af74dc275c8d3b7939a85e/lava_callback.json.gz DEBUG: Setup failed for 69af74dc275c8d3b7939a85e. Transitting job node result: incomplete -> fail 2026-03-10 01:49:13,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:49:13,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af74dc275c8d3b7939a85e HTTP/1.1" 200 None 2026-03-10 01:49:13,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:49:13,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 01:49:13,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:49:13,258 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af74dc275c8d3b7939a85e HTTP/1.1" 200 None 2026-03-10 01:49:13,337 - lava_callback - INFO - Completed processing callback for node 69af74dc275c8d3b7939a85e INFO: 172.20.0.1:44534 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44536 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:49:16,843 - lava_callback - INFO - Received callback for node 69af788f275c8d3b7939a8ec INFO: 172.20.0.1:44552 - "POST /node/69af788f275c8d3b7939a8ec HTTP/1.0" 202 Accepted 2026-03-10 01:49:16,849 - lava_callback - INFO - Starting async job submit for node 69af788f275c8d3b7939a8ec 2026-03-10 01:49:16,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:49:16,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af788f275c8d3b7939a8ec HTTP/1.1" 200 None 2026-03-10 01:49:16,869 - lava_callback - DEBUG - Processing node 69af788f275c8d3b7939a8ec: baseline-arm64 2026-03-10 01:49:16,876 - lava_callback - WARNING - Failed to upload log 2026-03-10 01:49:16,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:49:17,028 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:49:17,030 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af788f275c8d3b7939a8ec/lava-logs.yaml 2026-03-10 01:49:17,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:49:17,186 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:49:17,188 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af788f275c8d3b7939a8ec/lava_callback.json.gz 2026-03-10 01:49:17,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:49:17,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af788f275c8d3b7939a8ec HTTP/1.1" 200 None 2026-03-10 01:49:17,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:49:17,221 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 01:49:17,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:49:17,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af788f275c8d3b7939a8ec HTTP/1.1" 200 None 2026-03-10 01:49:17,249 - lava_callback - INFO - Completed processing callback for node 69af788f275c8d3b7939a8ec 2026-03-10 01:49:22,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:49:22,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:34134 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:49:27,449 - lava_callback - INFO - Received callback for node 69af7244275c8d3b7939a801 INFO: 172.20.0.1:34150 - "POST /node/69af7244275c8d3b7939a801 HTTP/1.0" 202 Accepted 2026-03-10 01:49:27,460 - lava_callback - INFO - Starting async job submit for node 69af7244275c8d3b7939a801 2026-03-10 01:49:27,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:49:27,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af7244275c8d3b7939a801 HTTP/1.1" 200 None 2026-03-10 01:49:27,569 - lava_callback - DEBUG - Processing node 69af7244275c8d3b7939a801: baseline-arm64 2026-03-10 01:49:28,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:49:28,144 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:49:28,146 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af7244275c8d3b7939a801/log.txt.gz 2026-03-10 01:49:28,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:49:28,608 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:49:28,611 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af7244275c8d3b7939a801/lava-logs.yaml 2026-03-10 01:49:28,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:49:28,753 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:49:28,755 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af7244275c8d3b7939a801/lava_callback.json.gz DEBUG: Setup failed for 69af7244275c8d3b7939a801. Transitting job node result: incomplete -> fail 2026-03-10 01:49:28,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:49:28,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af7244275c8d3b7939a801 HTTP/1.1" 200 None 2026-03-10 01:49:28,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:49:28,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 01:49:28,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:49:28,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af7244275c8d3b7939a801 HTTP/1.1" 200 None 2026-03-10 01:49:28,982 - lava_callback - INFO - Completed processing callback for node 69af7244275c8d3b7939a801 INFO: 172.20.0.1:49866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59518 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:49:52,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:49:52,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:51682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53000 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.20.0.1:53012 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37038 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:50:46,723 - lava_callback - INFO - Received callback for node 69af77b8275c8d3b7939a8cc INFO: 172.20.0.1:37042 - "POST /node/69af77b8275c8d3b7939a8cc HTTP/1.0" 202 Accepted 2026-03-10 01:50:46,729 - lava_callback - INFO - Starting async job submit for node 69af77b8275c8d3b7939a8cc 2026-03-10 01:50:46,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:50:46,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af77b8275c8d3b7939a8cc HTTP/1.1" 200 None 2026-03-10 01:50:46,769 - lava_callback - DEBUG - Processing node 69af77b8275c8d3b7939a8cc: baseline-arm64-foundriesio 2026-03-10 01:50:46,820 - lava_callback - WARNING - Failed to upload log 2026-03-10 01:50:46,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:50:47,022 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:50:47,024 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69af77b8275c8d3b7939a8cc/lava-logs.yaml 2026-03-10 01:50:47,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:50:47,156 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:50:47,158 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69af77b8275c8d3b7939a8cc/lava_callback.json.gz 2026-03-10 01:50:47,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:50:47,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af77b8275c8d3b7939a8cc HTTP/1.1" 200 None 2026-03-10 01:50:47,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:50:47,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 01:50:47,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:50:47,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af77b8275c8d3b7939a8cc HTTP/1.1" 200 None 2026-03-10 01:50:47,262 - lava_callback - INFO - Completed processing callback for node 69af77b8275c8d3b7939a8cc 2026-03-10 01:50:52,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:50:52,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:36476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51410 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56928 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56944 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:52:24,970 - lava_callback - INFO - Received callback for node 69af72ed275c8d3b7939a821 INFO: 172.20.0.1:37324 - "POST /node/69af72ed275c8d3b7939a821 HTTP/1.0" 202 Accepted 2026-03-10 01:52:24,977 - lava_callback - INFO - Starting async job submit for node 69af72ed275c8d3b7939a821 2026-03-10 01:52:25,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:52:25,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af72ed275c8d3b7939a821 HTTP/1.1" 200 None 2026-03-10 01:52:25,036 - lava_callback - DEBUG - Processing node 69af72ed275c8d3b7939a821: baseline-nfs-arm64 2026-03-10 01:52:25,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:52:25,596 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:52:25,597 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af72ed275c8d3b7939a821/log.txt.gz 2026-03-10 01:52:25,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:52:26,083 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:52:26,085 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af72ed275c8d3b7939a821/lava-logs.yaml 2026-03-10 01:52:26,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:52:26,255 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:52:26,257 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af72ed275c8d3b7939a821/lava_callback.json.gz DEBUG: Setup failed for 69af72ed275c8d3b7939a821. Transitting job node result: incomplete -> fail 2026-03-10 01:52:26,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:52:26,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af72ed275c8d3b7939a821 HTTP/1.1" 200 None 2026-03-10 01:52:26,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:52:26,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 01:52:26,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:52:26,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af72ed275c8d3b7939a821 HTTP/1.1" 200 None 2026-03-10 01:52:26,480 - lava_callback - INFO - Completed processing callback for node 69af72ed275c8d3b7939a821 INFO: 172.20.0.1:37330 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:52:27,106 - lava_callback - INFO - Received callback for node 69af77ff275c8d3b7939a8d1 INFO: 172.20.0.1:37344 - "POST /node/69af77ff275c8d3b7939a8d1 HTTP/1.0" 202 Accepted 2026-03-10 01:52:27,113 - lava_callback - INFO - Starting async job submit for node 69af77ff275c8d3b7939a8d1 2026-03-10 01:52:27,147 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:52:27,166 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af77ff275c8d3b7939a8d1 HTTP/1.1" 200 None 2026-03-10 01:52:27,167 - lava_callback - DEBUG - Processing node 69af77ff275c8d3b7939a8d1: baseline-arm64-broonie 2026-03-10 01:52:27,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:52:27,523 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:52:27,525 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af77ff275c8d3b7939a8d1/log.txt.gz 2026-03-10 01:52:27,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:52:27,879 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:52:27,881 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af77ff275c8d3b7939a8d1/lava-logs.yaml 2026-03-10 01:52:27,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:52:28,010 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:52:28,012 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af77ff275c8d3b7939a8d1/lava_callback.json.gz DEBUG: Setup failed for 69af77ff275c8d3b7939a8d1. Transitting job node result: incomplete -> fail 2026-03-10 01:52:28,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:52:28,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af77ff275c8d3b7939a8d1 HTTP/1.1" 200 None 2026-03-10 01:52:28,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:52:28,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 01:52:28,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:52:28,157 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af77ff275c8d3b7939a8d1 HTTP/1.1" 200 None 2026-03-10 01:52:28,239 - lava_callback - INFO - Completed processing callback for node 69af77ff275c8d3b7939a8d1 INFO: 172.20.0.1:50686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53406 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:52:52,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:52:52,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:59258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41658 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54372 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37970 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:54:29,335 - lava_callback - INFO - Received callback for node 69af7683275c8d3b7939a8a0 INFO: 172.20.0.1:37984 - "POST /node/69af7683275c8d3b7939a8a0 HTTP/1.0" 202 Accepted 2026-03-10 01:54:29,341 - lava_callback - INFO - Starting async job submit for node 69af7683275c8d3b7939a8a0 2026-03-10 01:54:29,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:54:29,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af7683275c8d3b7939a8a0 HTTP/1.1" 200 None 2026-03-10 01:54:29,392 - lava_callback - DEBUG - Processing node 69af7683275c8d3b7939a8a0: baseline-arm64-broonie 2026-03-10 01:54:29,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:54:29,963 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:54:29,964 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af7683275c8d3b7939a8a0/log.txt.gz 2026-03-10 01:54:30,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:54:30,403 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:54:30,406 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af7683275c8d3b7939a8a0/lava-logs.yaml 2026-03-10 01:54:30,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:54:30,560 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:54:30,561 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af7683275c8d3b7939a8a0/lava_callback.json.gz DEBUG: Setup failed for 69af7683275c8d3b7939a8a0. Transitting job node result: incomplete -> fail 2026-03-10 01:54:30,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:54:30,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af7683275c8d3b7939a8a0 HTTP/1.1" 200 None 2026-03-10 01:54:30,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:54:30,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 01:54:30,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:54:30,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af7683275c8d3b7939a8a0 HTTP/1.1" 200 None 2026-03-10 01:54:30,771 - lava_callback - INFO - Completed processing callback for node 69af7683275c8d3b7939a8a0 INFO: 172.20.0.1:52840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36338 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:54:52,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:54:52,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:57632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49848 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35586 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47536 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42106 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39512 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:58:57,265 - lava_callback - INFO - Received callback for node 69af7abd275c8d3b7939a971 INFO: 172.20.0.1:39518 - "POST /node/69af7abd275c8d3b7939a971 HTTP/1.0" 202 Accepted 2026-03-10 01:58:57,272 - lava_callback - INFO - Starting async job submit for node 69af7abd275c8d3b7939a971 2026-03-10 01:58:57,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:58:57,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af7abd275c8d3b7939a971 HTTP/1.1" 200 None 2026-03-10 01:58:57,304 - lava_callback - DEBUG - Processing node 69af7abd275c8d3b7939a971: baseline-arm64 2026-03-10 01:58:57,352 - lava_callback - WARNING - Failed to upload log 2026-03-10 01:58:57,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:58:57,545 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:58:57,546 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af7abd275c8d3b7939a971/lava-logs.yaml 2026-03-10 01:58:57,548 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 01:58:57,684 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 01:58:57,685 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af7abd275c8d3b7939a971/lava_callback.json.gz 2026-03-10 01:58:57,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:58:57,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af7abd275c8d3b7939a971 HTTP/1.1" 200 None 2026-03-10 01:58:57,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:58:57,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 01:58:57,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:58:57,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af7abd275c8d3b7939a971 HTTP/1.1" 200 None 2026-03-10 01:58:57,802 - lava_callback - INFO - Completed processing callback for node 69af7abd275c8d3b7939a971 INFO: 172.20.0.1:37252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53766 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53774 - "GET / HTTP/1.0" 200 OK 2026-03-10 01:59:22,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 01:59:22,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:56358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49782 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56112 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:01:10,807 - lava_callback - INFO - Received callback for node 69af7760275c8d3b7939a8bd INFO: 172.20.0.1:42500 - "POST /node/69af7760275c8d3b7939a8bd HTTP/1.0" 202 Accepted 2026-03-10 02:01:10,814 - lava_callback - INFO - Starting async job submit for node 69af7760275c8d3b7939a8bd 2026-03-10 02:01:10,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:01:10,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af7760275c8d3b7939a8bd HTTP/1.1" 200 None 2026-03-10 02:01:10,877 - lava_callback - DEBUG - Processing node 69af7760275c8d3b7939a8bd: baseline-arm64-broonie 2026-03-10 02:01:11,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:01:11,482 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:01:11,484 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af7760275c8d3b7939a8bd/log.txt.gz 2026-03-10 02:01:11,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:01:11,954 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:01:11,956 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af7760275c8d3b7939a8bd/lava-logs.yaml 2026-03-10 02:01:11,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:01:12,141 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:01:12,143 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af7760275c8d3b7939a8bd/lava_callback.json.gz DEBUG: Setup failed for 69af7760275c8d3b7939a8bd. Transitting job node result: incomplete -> fail 2026-03-10 02:01:12,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:01:12,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af7760275c8d3b7939a8bd HTTP/1.1" 200 None 2026-03-10 02:01:12,239 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:01:12,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 02:01:12,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:01:12,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af7760275c8d3b7939a8bd HTTP/1.1" 200 None 2026-03-10 02:01:12,348 - lava_callback - INFO - Completed processing callback for node 69af7760275c8d3b7939a8bd INFO: 172.20.0.1:42506 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42522 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:01:22,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:01:22,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:41436 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:01:34,376 - lava_callback - INFO - Received callback for node 69acd86bbcd207e915d77ccf INFO: 172.20.0.1:46108 - "POST /node/69acd86bbcd207e915d77ccf HTTP/1.0" 202 Accepted 2026-03-10 02:01:34,382 - lava_callback - INFO - Starting async job submit for node 69acd86bbcd207e915d77ccf 2026-03-10 02:01:34,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:01:34,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acd86bbcd207e915d77ccf HTTP/1.1" 200 None 2026-03-10 02:01:34,397 - lava_callback - DEBUG - Processing node 69acd86bbcd207e915d77ccf: baseline-arm-baylibre 2026-03-10 02:01:34,398 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acd86bbcd207e915d77ccf 2026-03-10 02:01:34,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:01:34,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acd86bbcd207e915d77ccf HTTP/1.1" 200 None 2026-03-10 02:01:34,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:01:34,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a4d6d27caa0b8e1b59d8ae HTTP/1.1" 200 None 2026-03-10 02:01:34,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:01:34,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acd86bbcd207e915d77ccf HTTP/1.1" 200 None 2026-03-10 02:01:34,453 - lava_callback - ERROR - Error processing callback for node 69acd86bbcd207e915d77ccf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58940 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:02:14,371 - lava_callback - INFO - Received callback for node 69acd894bcd207e915d77ce0 INFO: 172.20.0.1:45778 - "POST /node/69acd894bcd207e915d77ce0 HTTP/1.0" 202 Accepted 2026-03-10 02:02:14,377 - lava_callback - INFO - Starting async job submit for node 69acd894bcd207e915d77ce0 2026-03-10 02:02:14,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:02:14,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acd894bcd207e915d77ce0 HTTP/1.1" 200 None 2026-03-10 02:02:14,393 - lava_callback - DEBUG - Processing node 69acd894bcd207e915d77ce0: baseline-arm-baylibre 2026-03-10 02:02:14,393 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acd894bcd207e915d77ce0 2026-03-10 02:02:14,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:02:14,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acd894bcd207e915d77ce0 HTTP/1.1" 200 None 2026-03-10 02:02:14,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:02:14,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a4d6d27caa0b8e1b59d8ae HTTP/1.1" 200 None 2026-03-10 02:02:14,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:02:14,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acd894bcd207e915d77ce0 HTTP/1.1" 200 None 2026-03-10 02:02:14,476 - lava_callback - ERROR - Error processing callback for node 69acd894bcd207e915d77ce0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45794 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38216 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58494 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:04:14,523 - lava_callback - INFO - Received callback for node 69acd90dbcd207e915d77d0e INFO: 172.20.0.1:45290 - "POST /node/69acd90dbcd207e915d77d0e HTTP/1.0" 202 Accepted 2026-03-10 02:04:14,529 - lava_callback - INFO - Starting async job submit for node 69acd90dbcd207e915d77d0e 2026-03-10 02:04:14,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:04:14,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acd90dbcd207e915d77d0e HTTP/1.1" 200 None 2026-03-10 02:04:14,544 - lava_callback - DEBUG - Processing node 69acd90dbcd207e915d77d0e: baseline-arm-baylibre 2026-03-10 02:04:14,544 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acd90dbcd207e915d77d0e 2026-03-10 02:04:14,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:04:14,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acd90dbcd207e915d77d0e HTTP/1.1" 200 None 2026-03-10 02:04:14,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:04:14,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a4d6d27caa0b8e1b59d8ae HTTP/1.1" 200 None 2026-03-10 02:04:14,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:04:14,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acd90dbcd207e915d77d0e HTTP/1.1" 200 None 2026-03-10 02:04:14,601 - lava_callback - ERROR - Error processing callback for node 69acd90dbcd207e915d77d0e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45292 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58796 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:04:48,357 - lava_callback - INFO - Received callback for node 69af738d275c8d3b7939a830 INFO: 172.20.0.1:58808 - "POST /node/69af738d275c8d3b7939a830 HTTP/1.0" 202 Accepted 2026-03-10 02:04:48,364 - lava_callback - INFO - Starting async job submit for node 69af738d275c8d3b7939a830 2026-03-10 02:04:48,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:04:48,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af738d275c8d3b7939a830 HTTP/1.1" 200 None 2026-03-10 02:04:48,410 - lava_callback - DEBUG - Processing node 69af738d275c8d3b7939a830: baseline-arm64 2026-03-10 02:04:49,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:04:49,413 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:04:49,415 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af738d275c8d3b7939a830/log.txt.gz 2026-03-10 02:04:49,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:04:50,165 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:04:50,167 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af738d275c8d3b7939a830/lava-logs.yaml 2026-03-10 02:04:50,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:04:50,360 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:04:50,362 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af738d275c8d3b7939a830/lava_callback.json.gz DEBUG: Setup failed for 69af738d275c8d3b7939a830. Transitting job node result: incomplete -> fail 2026-03-10 02:04:50,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:04:50,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af738d275c8d3b7939a830 HTTP/1.1" 200 None 2026-03-10 02:04:50,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:04:50,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 02:04:50,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:04:50,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af738d275c8d3b7939a830 HTTP/1.1" 200 None 2026-03-10 02:04:50,605 - lava_callback - INFO - Completed processing callback for node 69af738d275c8d3b7939a830 2026-03-10 02:04:52,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:04:52,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-10 02:04:54,503 - lava_callback - INFO - Received callback for node 69acd934bcd207e915d77d1e INFO: 172.20.0.1:47056 - "POST /node/69acd934bcd207e915d77d1e HTTP/1.0" 202 Accepted 2026-03-10 02:04:54,508 - lava_callback - INFO - Starting async job submit for node 69acd934bcd207e915d77d1e 2026-03-10 02:04:54,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:04:54,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acd934bcd207e915d77d1e HTTP/1.1" 200 None 2026-03-10 02:04:54,529 - lava_callback - DEBUG - Processing node 69acd934bcd207e915d77d1e: baseline-arm-baylibre 2026-03-10 02:04:54,529 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acd934bcd207e915d77d1e 2026-03-10 02:04:54,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:04:54,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acd934bcd207e915d77d1e HTTP/1.1" 200 None 2026-03-10 02:04:54,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:04:54,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a4d6d27caa0b8e1b59d8ae HTTP/1.1" 200 None 2026-03-10 02:04:54,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:04:54,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acd934bcd207e915d77d1e HTTP/1.1" 200 None 2026-03-10 02:04:54,593 - lava_callback - ERROR - Error processing callback for node 69acd934bcd207e915d77d1e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38032 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39782 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:05:34,646 - lava_callback - INFO - Received callback for node 69acd95cbcd207e915d77d2f INFO: 172.20.0.1:41136 - "POST /node/69acd95cbcd207e915d77d2f HTTP/1.0" 202 Accepted 2026-03-10 02:05:34,652 - lava_callback - INFO - Starting async job submit for node 69acd95cbcd207e915d77d2f 2026-03-10 02:05:34,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:05:34,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acd95cbcd207e915d77d2f HTTP/1.1" 200 None 2026-03-10 02:05:34,667 - lava_callback - DEBUG - Processing node 69acd95cbcd207e915d77d2f: baseline-arm-baylibre 2026-03-10 02:05:34,667 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acd95cbcd207e915d77d2f 2026-03-10 02:05:34,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:05:34,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acd95cbcd207e915d77d2f HTTP/1.1" 200 None 2026-03-10 02:05:34,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:05:34,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a4d6d27caa0b8e1b59d8ae HTTP/1.1" 200 None 2026-03-10 02:05:34,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:05:34,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acd95cbcd207e915d77d2f HTTP/1.1" 200 None 2026-03-10 02:05:34,718 - lava_callback - ERROR - Error processing callback for node 69acd95cbcd207e915d77d2f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55532 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:06:01,417 - lava_callback - INFO - Received callback for node 69af7439275c8d3b7939a845 INFO: 172.20.0.1:36894 - "POST /node/69af7439275c8d3b7939a845 HTTP/1.0" 202 Accepted 2026-03-10 02:06:01,425 - lava_callback - INFO - Starting async job submit for node 69af7439275c8d3b7939a845 2026-03-10 02:06:01,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:06:01,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af7439275c8d3b7939a845 HTTP/1.1" 200 None 2026-03-10 02:06:01,473 - lava_callback - DEBUG - Processing node 69af7439275c8d3b7939a845: baseline-nfs-arm64 2026-03-10 02:06:02,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:06:02,457 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:06:02,458 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af7439275c8d3b7939a845/log.txt.gz 2026-03-10 02:06:02,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:06:03,159 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:06:03,162 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af7439275c8d3b7939a845/lava-logs.yaml 2026-03-10 02:06:03,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:06:03,347 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:06:03,349 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af7439275c8d3b7939a845/lava_callback.json.gz DEBUG: Setup failed for 69af7439275c8d3b7939a845. Transitting job node result: incomplete -> fail 2026-03-10 02:06:03,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:06:03,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af7439275c8d3b7939a845 HTTP/1.1" 200 None 2026-03-10 02:06:03,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:06:03,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 02:06:03,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:06:03,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af7439275c8d3b7939a845 HTTP/1.1" 200 None 2026-03-10 02:06:03,530 - lava_callback - INFO - Completed processing callback for node 69af7439275c8d3b7939a845 INFO: 172.20.0.1:36906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36918 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:06:14,616 - lava_callback - INFO - Received callback for node 69acd983bcd207e915d77d3d INFO: 172.20.0.1:53978 - "POST /node/69acd983bcd207e915d77d3d HTTP/1.0" 202 Accepted 2026-03-10 02:06:14,622 - lava_callback - INFO - Starting async job submit for node 69acd983bcd207e915d77d3d 2026-03-10 02:06:14,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:06:14,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acd983bcd207e915d77d3d HTTP/1.1" 200 None 2026-03-10 02:06:14,641 - lava_callback - DEBUG - Processing node 69acd983bcd207e915d77d3d: baseline-arm-baylibre 2026-03-10 02:06:14,642 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acd983bcd207e915d77d3d 2026-03-10 02:06:14,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:06:14,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acd983bcd207e915d77d3d HTTP/1.1" 200 None 2026-03-10 02:06:14,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:06:14,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a4d6d27caa0b8e1b59d8ae HTTP/1.1" 200 None 2026-03-10 02:06:14,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:06:14,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acd983bcd207e915d77d3d HTTP/1.1" 200 None 2026-03-10 02:06:14,697 - lava_callback - ERROR - Error processing callback for node 69acd983bcd207e915d77d3d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53990 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53994 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:06:22,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:06:22,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:49600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56432 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47146 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60144 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60156 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:09:22,948 - lava_callback - INFO - Received callback for node 69af7d37275c8d3b7939a9ec INFO: 172.20.0.1:59228 - "POST /node/69af7d37275c8d3b7939a9ec HTTP/1.0" 202 Accepted 2026-03-10 02:09:22,954 - lava_callback - INFO - Starting async job submit for node 69af7d37275c8d3b7939a9ec 2026-03-10 02:09:22,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:09:22,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af7d37275c8d3b7939a9ec HTTP/1.1" 200 None 2026-03-10 02:09:22,989 - lava_callback - DEBUG - Processing node 69af7d37275c8d3b7939a9ec: baseline-arm64 2026-03-10 02:09:23,039 - lava_callback - WARNING - Failed to upload log 2026-03-10 02:09:23,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:09:23,232 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:09:23,234 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af7d37275c8d3b7939a9ec/lava-logs.yaml 2026-03-10 02:09:23,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:09:23,379 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:09:23,381 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af7d37275c8d3b7939a9ec/lava_callback.json.gz 2026-03-10 02:09:23,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:09:23,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af7d37275c8d3b7939a9ec HTTP/1.1" 200 None 2026-03-10 02:09:23,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:09:23,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 02:09:23,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:09:23,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af7d37275c8d3b7939a9ec HTTP/1.1" 200 None 2026-03-10 02:09:23,498 - lava_callback - INFO - Completed processing callback for node 69af7d37275c8d3b7939a9ec INFO: 172.20.0.1:59244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50490 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:09:52,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:09:52,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:33572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44360 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44940 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42598 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42612 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:12:19,584 - lava_callback - INFO - Received callback for node 69af7de0275c8d3b7939aa0b INFO: 172.20.0.1:42628 - "POST /node/69af7de0275c8d3b7939aa0b HTTP/1.0" 202 Accepted 2026-03-10 02:12:19,591 - lava_callback - INFO - Starting async job submit for node 69af7de0275c8d3b7939aa0b 2026-03-10 02:12:19,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:12:19,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af7de0275c8d3b7939aa0b HTTP/1.1" 200 None 2026-03-10 02:12:19,617 - lava_callback - DEBUG - Processing node 69af7de0275c8d3b7939aa0b: baseline-nfs-arm64 2026-03-10 02:12:19,619 - lava_callback - WARNING - Failed to upload log 2026-03-10 02:12:19,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:12:19,756 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:12:19,758 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af7de0275c8d3b7939aa0b/lava-logs.yaml 2026-03-10 02:12:19,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:12:19,933 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:12:19,934 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af7de0275c8d3b7939aa0b/lava_callback.json.gz 2026-03-10 02:12:19,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:12:19,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af7de0275c8d3b7939aa0b HTTP/1.1" 200 None 2026-03-10 02:12:19,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:12:19,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 02:12:19,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:12:19,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af7de0275c8d3b7939aa0b HTTP/1.1" 200 None 2026-03-10 02:12:19,991 - lava_callback - INFO - Completed processing callback for node 69af7de0275c8d3b7939aa0b 2026-03-10 02:12:22,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:12:22,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:48784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57068 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46112 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:13:38,708 - lava_callback - INFO - Received callback for node 69af7e2b275c8d3b7939aa1c INFO: 172.20.0.1:46124 - "POST /node/69af7e2b275c8d3b7939aa1c HTTP/1.0" 202 Accepted 2026-03-10 02:13:38,715 - lava_callback - INFO - Starting async job submit for node 69af7e2b275c8d3b7939aa1c 2026-03-10 02:13:38,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:13:38,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af7e2b275c8d3b7939aa1c HTTP/1.1" 200 None 2026-03-10 02:13:38,747 - lava_callback - DEBUG - Processing node 69af7e2b275c8d3b7939aa1c: baseline-arm64 2026-03-10 02:13:38,798 - lava_callback - WARNING - Failed to upload log 2026-03-10 02:13:38,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:13:39,024 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:13:39,025 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af7e2b275c8d3b7939aa1c/lava-logs.yaml 2026-03-10 02:13:39,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:13:39,159 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:13:39,161 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af7e2b275c8d3b7939aa1c/lava_callback.json.gz 2026-03-10 02:13:39,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:13:39,193 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af7e2b275c8d3b7939aa1c HTTP/1.1" 200 None 2026-03-10 02:13:39,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:13:39,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 02:13:39,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:13:39,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af7e2b275c8d3b7939aa1c HTTP/1.1" 200 None 2026-03-10 02:13:39,295 - lava_callback - INFO - Completed processing callback for node 69af7e2b275c8d3b7939aa1c 2026-03-10 02:13:46,232 - lava_callback - INFO - Received callback for node 69af7bae275c8d3b7939a99c INFO: 172.20.0.1:55578 - "POST /node/69af7bae275c8d3b7939a99c HTTP/1.0" 202 Accepted 2026-03-10 02:13:46,238 - lava_callback - INFO - Starting async job submit for node 69af7bae275c8d3b7939a99c 2026-03-10 02:13:46,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:13:46,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af7bae275c8d3b7939a99c HTTP/1.1" 200 None 2026-03-10 02:13:46,284 - lava_callback - DEBUG - Processing node 69af7bae275c8d3b7939a99c: baseline-nfs-arm64 INFO: 172.20.0.1:55588 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:13:46,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:13:46,786 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:13:46,788 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af7bae275c8d3b7939a99c/log.txt.gz 2026-03-10 02:13:46,971 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:13:47,194 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:13:47,196 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af7bae275c8d3b7939a99c/lava-logs.yaml 2026-03-10 02:13:47,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:13:47,386 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:13:47,388 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af7bae275c8d3b7939a99c/lava_callback.json.gz 2026-03-10 02:13:47,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:13:47,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af7bae275c8d3b7939a99c HTTP/1.1" 200 None 2026-03-10 02:13:47,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:13:47,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 02:13:47,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:13:47,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af7bae275c8d3b7939a99c HTTP/1.1" 200 None 2026-03-10 02:13:47,564 - lava_callback - INFO - Completed processing callback for node 69af7bae275c8d3b7939a99c 2026-03-10 02:13:52,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:13:52,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:42746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39640 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52168 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:15:03,193 - lava_callback - INFO - Received callback for node 69af7ca9275c8d3b7939a9d8 INFO: 172.20.0.1:34644 - "POST /node/69af7ca9275c8d3b7939a9d8 HTTP/1.0" 202 Accepted 2026-03-10 02:15:03,200 - lava_callback - INFO - Starting async job submit for node 69af7ca9275c8d3b7939a9d8 2026-03-10 02:15:03,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:15:03,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af7ca9275c8d3b7939a9d8 HTTP/1.1" 200 None 2026-03-10 02:15:03,250 - lava_callback - DEBUG - Processing node 69af7ca9275c8d3b7939a9d8: baseline-arm64-broonie 2026-03-10 02:15:03,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:15:03,799 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:15:03,801 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af7ca9275c8d3b7939a9d8/log.txt.gz 2026-03-10 02:15:04,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:15:04,164 - lava_callback - INFO - Received callback for node 69af7b5b275c8d3b7939a988 INFO: 172.20.0.1:34658 - "POST /node/69af7b5b275c8d3b7939a988 HTTP/1.0" 202 Accepted 2026-03-10 02:15:04,170 - lava_callback - INFO - Starting async job submit for node 69af7b5b275c8d3b7939a988 2026-03-10 02:15:04,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:15:04,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af7b5b275c8d3b7939a988 HTTP/1.1" 200 None 2026-03-10 02:15:04,219 - lava_callback - DEBUG - Processing node 69af7b5b275c8d3b7939a988: baseline-arm64 2026-03-10 02:15:04,278 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:15:04,321 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af7ca9275c8d3b7939a9d8/lava-logs.yaml INFO: 172.20.0.1:34662 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:15:04,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:15:04,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:15:04,616 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:15:04,618 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af7b5b275c8d3b7939a988/log.txt.gz 2026-03-10 02:15:04,680 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:15:04,713 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af7ca9275c8d3b7939a9d8/lava_callback.json.gz DEBUG: Setup failed for 69af7ca9275c8d3b7939a9d8. Transitting job node result: incomplete -> fail 2026-03-10 02:15:04,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:15:04,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:15:04,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af7ca9275c8d3b7939a9d8 HTTP/1.1" 200 None 2026-03-10 02:15:04,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:15:04,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 02:15:04,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:15:04,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af7ca9275c8d3b7939a9d8 HTTP/1.1" 200 None 2026-03-10 02:15:05,040 - lava_callback - INFO - Completed processing callback for node 69af7ca9275c8d3b7939a9d8 2026-03-10 02:15:05,154 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:15:05,156 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af7b5b275c8d3b7939a988/lava-logs.yaml 2026-03-10 02:15:05,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:15:05,345 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:15:05,347 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af7b5b275c8d3b7939a988/lava_callback.json.gz 2026-03-10 02:15:05,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:15:05,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af7b5b275c8d3b7939a988 HTTP/1.1" 200 None 2026-03-10 02:15:05,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:15:05,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 02:15:05,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:15:05,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af7b5b275c8d3b7939a988 HTTP/1.1" 200 None 2026-03-10 02:15:05,525 - lava_callback - INFO - Completed processing callback for node 69af7b5b275c8d3b7939a988 INFO: 172.20.0.1:34676 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:15:09,695 - lava_callback - INFO - Received callback for node 69af7e8d275c8d3b7939aa34 INFO: 172.20.0.1:34684 - "POST /node/69af7e8d275c8d3b7939aa34 HTTP/1.0" 202 Accepted 2026-03-10 02:15:09,700 - lava_callback - INFO - Starting async job submit for node 69af7e8d275c8d3b7939aa34 2026-03-10 02:15:09,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:15:09,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af7e8d275c8d3b7939aa34 HTTP/1.1" 200 None 2026-03-10 02:15:09,719 - lava_callback - DEBUG - Processing node 69af7e8d275c8d3b7939aa34: baseline-arm64-broonie 2026-03-10 02:15:09,724 - lava_callback - WARNING - Failed to upload log 2026-03-10 02:15:09,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:15:09,870 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:15:09,872 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af7e8d275c8d3b7939aa34/lava-logs.yaml 2026-03-10 02:15:09,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:15:09,993 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:15:09,996 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af7e8d275c8d3b7939aa34/lava_callback.json.gz 2026-03-10 02:15:10,000 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:15:10,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af7e8d275c8d3b7939aa34 HTTP/1.1" 200 None 2026-03-10 02:15:10,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:15:10,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 02:15:10,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:15:10,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af7e8d275c8d3b7939aa34 HTTP/1.1" 200 None 2026-03-10 02:15:10,066 - lava_callback - INFO - Completed processing callback for node 69af7e8d275c8d3b7939aa34 INFO: 172.20.0.1:35642 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35650 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:15:22,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:15:22,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:36016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57660 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:15:43,083 - lava_callback - INFO - Received callback for node 69af75d8275c8d3b7939a88b INFO: 172.20.0.1:48634 - "POST /node/69af75d8275c8d3b7939a88b HTTP/1.0" 202 Accepted 2026-03-10 02:15:43,089 - lava_callback - INFO - Starting async job submit for node 69af75d8275c8d3b7939a88b 2026-03-10 02:15:43,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:15:43,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af75d8275c8d3b7939a88b HTTP/1.1" 200 None 2026-03-10 02:15:43,142 - lava_callback - DEBUG - Processing node 69af75d8275c8d3b7939a88b: baseline-arm64-broonie 2026-03-10 02:15:43,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:15:43,488 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:15:43,490 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af75d8275c8d3b7939a88b/log.txt.gz 2026-03-10 02:15:43,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:15:43,828 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:15:43,830 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af75d8275c8d3b7939a88b/lava-logs.yaml 2026-03-10 02:15:43,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:15:43,987 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:15:43,989 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af75d8275c8d3b7939a88b/lava_callback.json.gz DEBUG: Setup failed for 69af75d8275c8d3b7939a88b. Transitting job node result: incomplete -> fail 2026-03-10 02:15:44,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:15:44,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af75d8275c8d3b7939a88b HTTP/1.1" 200 None 2026-03-10 02:15:44,076 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:15:44,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 02:15:44,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:15:44,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af75d8275c8d3b7939a88b HTTP/1.1" 200 None 2026-03-10 02:15:44,191 - lava_callback - INFO - Completed processing callback for node 69af75d8275c8d3b7939a88b INFO: 172.20.0.1:48646 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:15:52,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:15:52,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:49614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33286 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:16:10,975 - lava_callback - INFO - Received callback for node 69af7edb275c8d3b7939aa4f INFO: 172.20.0.1:36824 - "POST /node/69af7edb275c8d3b7939aa4f HTTP/1.0" 202 Accepted 2026-03-10 02:16:10,981 - lava_callback - INFO - Starting async job submit for node 69af7edb275c8d3b7939aa4f 2026-03-10 02:16:10,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:16:10,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af7edb275c8d3b7939aa4f HTTP/1.1" 200 None 2026-03-10 02:16:10,998 - lava_callback - DEBUG - Processing node 69af7edb275c8d3b7939aa4f: baseline-arm64-clabbe 2026-03-10 02:16:11,002 - lava_callback - WARNING - Failed to upload log 2026-03-10 02:16:11,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:16:11,163 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:16:11,165 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69af7edb275c8d3b7939aa4f/lava-logs.yaml 2026-03-10 02:16:11,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:16:11,312 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:16:11,314 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69af7edb275c8d3b7939aa4f/lava_callback.json.gz 2026-03-10 02:16:11,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:16:11,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af7edb275c8d3b7939aa4f HTTP/1.1" 200 None 2026-03-10 02:16:11,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:16:11,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 02:16:11,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:16:11,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af7edb275c8d3b7939aa4f HTTP/1.1" 200 None 2026-03-10 02:16:11,376 - lava_callback - INFO - Completed processing callback for node 69af7edb275c8d3b7939aa4f INFO: 172.20.0.1:36828 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36836 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:16:22,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:16:22,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:33596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59588 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34488 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39658 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37034 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:19:30,473 - lava_callback - INFO - Received callback for node 69af7b15275c8d3b7939a97f INFO: 172.20.0.1:52428 - "POST /node/69af7b15275c8d3b7939a97f HTTP/1.0" 202 Accepted 2026-03-10 02:19:30,480 - lava_callback - INFO - Starting async job submit for node 69af7b15275c8d3b7939a97f 2026-03-10 02:19:30,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:19:30,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af7b15275c8d3b7939a97f HTTP/1.1" 200 None 2026-03-10 02:19:30,521 - lava_callback - DEBUG - Processing node 69af7b15275c8d3b7939a97f: baseline-nfs-arm64 2026-03-10 02:19:31,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:19:31,193 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:19:31,195 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af7b15275c8d3b7939a97f/log.txt.gz 2026-03-10 02:19:31,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:19:31,701 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:19:31,704 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af7b15275c8d3b7939a97f/lava-logs.yaml 2026-03-10 02:19:31,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:19:32,937 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:19:32,939 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af7b15275c8d3b7939a97f/lava_callback.json.gz 2026-03-10 02:19:32,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:19:32,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af7b15275c8d3b7939a97f HTTP/1.1" 200 None 2026-03-10 02:19:32,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:19:32,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 02:19:32,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:19:33,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af7b15275c8d3b7939a97f HTTP/1.1" 200 None 2026-03-10 02:19:33,081 - lava_callback - INFO - Completed processing callback for node 69af7b15275c8d3b7939a97f INFO: 172.20.0.1:52442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57510 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:19:52,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:19:52,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:43424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59466 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-10 02:20:15,441 - lava_callback - INFO - Received callback for node 69acdccabcd207e915d77e1a INFO: 172.20.0.1:59474 - "POST /node/69acdccabcd207e915d77e1a HTTP/1.0" 202 Accepted 2026-03-10 02:20:15,446 - lava_callback - INFO - Starting async job submit for node 69acdccabcd207e915d77e1a 2026-03-10 02:20:15,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:20:15,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acdccabcd207e915d77e1a HTTP/1.1" 200 None 2026-03-10 02:20:15,462 - lava_callback - DEBUG - Processing node 69acdccabcd207e915d77e1a: baseline-arm64-baylibre 2026-03-10 02:20:15,463 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acdccabcd207e915d77e1a 2026-03-10 02:20:15,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:20:15,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acdccabcd207e915d77e1a HTTP/1.1" 200 None 2026-03-10 02:20:15,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:20:15,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a4d6e67caa0b8e1b59d8c1 HTTP/1.1" 200 None 2026-03-10 02:20:15,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:20:15,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acdccabcd207e915d77e1a HTTP/1.1" 200 None 2026-03-10 02:20:15,516 - lava_callback - ERROR - Error processing callback for node 69acdccabcd207e915d77e1a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36792 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:20:33,348 - lava_callback - INFO - Received callback for node 69af7f85275c8d3b7939aa65 INFO: 172.20.0.1:55692 - "POST /node/69af7f85275c8d3b7939aa65 HTTP/1.0" 202 Accepted 2026-03-10 02:20:33,354 - lava_callback - INFO - Starting async job submit for node 69af7f85275c8d3b7939aa65 2026-03-10 02:20:33,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:20:33,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af7f85275c8d3b7939aa65 HTTP/1.1" 200 None 2026-03-10 02:20:33,407 - lava_callback - DEBUG - Processing node 69af7f85275c8d3b7939aa65: baseline-arm64 2026-03-10 02:20:33,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:20:33,857 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:20:33,859 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af7f85275c8d3b7939aa65/log.txt.gz 2026-03-10 02:20:34,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:20:34,260 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:20:34,262 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af7f85275c8d3b7939aa65/lava-logs.yaml 2026-03-10 02:20:34,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:20:34,418 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:20:34,419 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af7f85275c8d3b7939aa65/lava_callback.json.gz DEBUG: Setup failed for 69af7f85275c8d3b7939aa65. Transitting job node result: incomplete -> fail 2026-03-10 02:20:34,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:20:34,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af7f85275c8d3b7939aa65 HTTP/1.1" 200 None 2026-03-10 02:20:34,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:20:34,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 02:20:34,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:20:34,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af7f85275c8d3b7939aa65 HTTP/1.1" 200 None 2026-03-10 02:20:34,614 - lava_callback - INFO - Completed processing callback for node 69af7f85275c8d3b7939aa65 INFO: 172.20.0.1:55708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37400 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:20:52,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:20:52,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:39606 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:21:04,083 - lava_callback - INFO - Received callback for node 69af7fe4275c8d3b7939aa79 INFO: 172.20.0.1:48022 - "POST /node/69af7fe4275c8d3b7939aa79 HTTP/1.0" 202 Accepted 2026-03-10 02:21:04,089 - lava_callback - INFO - Starting async job submit for node 69af7fe4275c8d3b7939aa79 2026-03-10 02:21:04,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:21:04,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af7fe4275c8d3b7939aa79 HTTP/1.1" 200 None 2026-03-10 02:21:04,107 - lava_callback - DEBUG - Processing node 69af7fe4275c8d3b7939aa79: baseline-arm64-broonie 2026-03-10 02:21:04,112 - lava_callback - WARNING - Failed to upload log 2026-03-10 02:21:04,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:21:04,250 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:21:04,251 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af7fe4275c8d3b7939aa79/lava-logs.yaml 2026-03-10 02:21:04,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:21:04,370 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:21:04,372 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af7fe4275c8d3b7939aa79/lava_callback.json.gz 2026-03-10 02:21:04,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:21:04,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af7fe4275c8d3b7939aa79 HTTP/1.1" 200 None 2026-03-10 02:21:04,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:21:04,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 02:21:04,405 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:21:04,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af7fe4275c8d3b7939aa79 HTTP/1.1" 200 None 2026-03-10 02:21:04,438 - lava_callback - INFO - Completed processing callback for node 69af7fe4275c8d3b7939aa79 INFO: 172.20.0.1:48028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44182 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44194 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:21:22,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:21:22,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:38574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60924 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49502 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40858 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51734 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43106 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50528 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32808 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33604 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34032 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58098 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58114 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:31:17,730 - lava_callback - INFO - Received callback for node 69af7cea275c8d3b7939a9df INFO: 172.20.0.1:58128 - "POST /node/69af7cea275c8d3b7939a9df HTTP/1.0" 202 Accepted 2026-03-10 02:31:17,737 - lava_callback - INFO - Starting async job submit for node 69af7cea275c8d3b7939a9df 2026-03-10 02:31:17,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:31:17,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af7cea275c8d3b7939a9df HTTP/1.1" 200 None 2026-03-10 02:31:17,776 - lava_callback - DEBUG - Processing node 69af7cea275c8d3b7939a9df: baseline-arm64 2026-03-10 02:31:18,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:31:18,444 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:31:18,446 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af7cea275c8d3b7939a9df/log.txt.gz 2026-03-10 02:31:18,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:31:18,961 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:31:18,963 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af7cea275c8d3b7939a9df/lava-logs.yaml 2026-03-10 02:31:18,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:31:19,158 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:31:19,160 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af7cea275c8d3b7939a9df/lava_callback.json.gz 2026-03-10 02:31:19,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:31:19,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af7cea275c8d3b7939a9df HTTP/1.1" 200 None 2026-03-10 02:31:19,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:31:19,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 02:31:19,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:31:19,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af7cea275c8d3b7939a9df HTTP/1.1" 200 None 2026-03-10 02:31:19,313 - lava_callback - INFO - Completed processing callback for node 69af7cea275c8d3b7939a9df 2026-03-10 02:31:22,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:31:23,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:40948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58964 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:31:39,711 - lava_callback - INFO - Received callback for node 69af823b275c8d3b7939aacb INFO: 172.20.0.1:55440 - "POST /node/69af823b275c8d3b7939aacb HTTP/1.0" 202 Accepted 2026-03-10 02:31:39,718 - lava_callback - INFO - Starting async job submit for node 69af823b275c8d3b7939aacb 2026-03-10 02:31:39,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:31:39,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af823b275c8d3b7939aacb HTTP/1.1" 200 None 2026-03-10 02:31:39,768 - lava_callback - DEBUG - Processing node 69af823b275c8d3b7939aacb: baseline-arm64 2026-03-10 02:31:40,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:31:40,378 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:31:40,380 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af823b275c8d3b7939aacb/log.txt.gz 2026-03-10 02:31:40,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:31:40,873 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:31:40,875 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af823b275c8d3b7939aacb/lava-logs.yaml 2026-03-10 02:31:40,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:31:41,007 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:31:41,009 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af823b275c8d3b7939aacb/lava_callback.json.gz DEBUG: Setup failed for 69af823b275c8d3b7939aacb. Transitting job node result: incomplete -> fail 2026-03-10 02:31:41,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:31:41,093 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af823b275c8d3b7939aacb HTTP/1.1" 200 None 2026-03-10 02:31:41,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:31:41,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 02:31:41,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:31:41,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af823b275c8d3b7939aacb HTTP/1.1" 200 None 2026-03-10 02:31:41,210 - lava_callback - INFO - Completed processing callback for node 69af823b275c8d3b7939aacb INFO: 172.20.0.1:55450 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:31:53,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:31:53,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:33220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57166 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:32:05,141 - lava_callback - INFO - Received callback for node 69af7e75275c8d3b7939aa2d INFO: 172.20.0.1:57170 - "POST /node/69af7e75275c8d3b7939aa2d HTTP/1.0" 202 Accepted 2026-03-10 02:32:05,148 - lava_callback - INFO - Starting async job submit for node 69af7e75275c8d3b7939aa2d 2026-03-10 02:32:05,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:32:05,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af7e75275c8d3b7939aa2d HTTP/1.1" 200 None 2026-03-10 02:32:05,245 - lava_callback - DEBUG - Processing node 69af7e75275c8d3b7939aa2d: ltp-smoke 2026-03-10 02:32:05,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:32:05,852 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:32:05,854 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69af7e75275c8d3b7939aa2d/log.txt.gz 2026-03-10 02:32:06,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:32:06,375 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:32:06,377 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69af7e75275c8d3b7939aa2d/lava-logs.yaml 2026-03-10 02:32:06,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:57172 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:32:06,558 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:32:06,560 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69af7e75275c8d3b7939aa2d/lava_callback.json.gz 2026-03-10 02:32:06,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:32:06,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af7e75275c8d3b7939aa2d HTTP/1.1" 200 None 2026-03-10 02:32:06,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:32:06,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6334275c8d3b79399dba HTTP/1.1" 200 None 2026-03-10 02:32:06,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:32:06,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af7e75275c8d3b7939aa2d HTTP/1.1" 200 None 2026-03-10 02:32:06,869 - lava_callback - INFO - Completed processing callback for node 69af7e75275c8d3b7939aa2d INFO: 172.20.0.1:53168 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53174 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:32:23,030 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:32:23,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:41986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36080 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:33:08,918 - lava_callback - INFO - Received callback for node 69af828b275c8d3b7939aad6 INFO: 172.20.0.1:36086 - "POST /node/69af828b275c8d3b7939aad6 HTTP/1.0" 202 Accepted 2026-03-10 02:33:08,924 - lava_callback - INFO - Starting async job submit for node 69af828b275c8d3b7939aad6 2026-03-10 02:33:08,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:33:08,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af828b275c8d3b7939aad6 HTTP/1.1" 200 None 2026-03-10 02:33:08,970 - lava_callback - DEBUG - Processing node 69af828b275c8d3b7939aad6: baseline-arm64 2026-03-10 02:33:09,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:33:09,475 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:33:09,477 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af828b275c8d3b7939aad6/log.txt.gz 2026-03-10 02:33:09,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:33:09,923 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:33:09,924 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af828b275c8d3b7939aad6/lava-logs.yaml 2026-03-10 02:33:09,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:33:10,064 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:33:10,066 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af828b275c8d3b7939aad6/lava_callback.json.gz DEBUG: Setup failed for 69af828b275c8d3b7939aad6. Transitting job node result: incomplete -> fail 2026-03-10 02:33:10,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:33:10,170 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af828b275c8d3b7939aad6 HTTP/1.1" 200 None 2026-03-10 02:33:10,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:33:10,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 02:33:10,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:33:10,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af828b275c8d3b7939aad6 HTTP/1.1" 200 None 2026-03-10 02:33:10,286 - lava_callback - INFO - Completed processing callback for node 69af828b275c8d3b7939aad6 INFO: 172.20.0.1:45364 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45370 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:33:23,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:33:23,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:55570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38680 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:34:11,564 - lava_callback - INFO - Received callback for node 69af7f7c275c8d3b7939aa62 INFO: 172.20.0.1:49104 - "POST /node/69af7f7c275c8d3b7939aa62 HTTP/1.0" 202 Accepted 2026-03-10 02:34:11,570 - lava_callback - INFO - Starting async job submit for node 69af7f7c275c8d3b7939aa62 2026-03-10 02:34:11,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:34:11,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af7f7c275c8d3b7939aa62 HTTP/1.1" 200 None 2026-03-10 02:34:11,642 - lava_callback - DEBUG - Processing node 69af7f7c275c8d3b7939aa62: baseline-x86 2026-03-10 02:34:12,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:34:12,539 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:34:12,541 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69af7f7c275c8d3b7939aa62/log.txt.gz 2026-03-10 02:34:12,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:34:13,230 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:34:13,233 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69af7f7c275c8d3b7939aa62/lava-logs.yaml 2026-03-10 02:34:13,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:34:13,416 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:34:13,418 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69af7f7c275c8d3b7939aa62/lava_callback.json.gz DEBUG: Setup failed for 69af7f7c275c8d3b7939aa62. Transitting job node result: incomplete -> fail 2026-03-10 02:34:13,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:34:13,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af7f7c275c8d3b7939aa62 HTTP/1.1" 200 None 2026-03-10 02:34:13,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:34:13,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af635a275c8d3b79399ddd HTTP/1.1" 200 None 2026-03-10 02:34:13,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:34:13,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af7f7c275c8d3b7939aa62 HTTP/1.1" 200 None 2026-03-10 02:34:13,674 - lava_callback - INFO - Completed processing callback for node 69af7f7c275c8d3b7939aa62 INFO: 172.20.0.1:49108 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49122 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:34:23,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:34:23,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:40714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38272 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:34:50,962 - lava_callback - INFO - Received callback for node 69af7f13275c8d3b7939aa59 INFO: 172.20.0.1:38242 - "POST /node/69af7f13275c8d3b7939aa59 HTTP/1.0" 202 Accepted 2026-03-10 02:34:50,969 - lava_callback - INFO - Starting async job submit for node 69af7f13275c8d3b7939aa59 2026-03-10 02:34:51,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:34:51,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af7f13275c8d3b7939aa59 HTTP/1.1" 200 None 2026-03-10 02:34:51,038 - lava_callback - DEBUG - Processing node 69af7f13275c8d3b7939aa59: baseline-arm 2026-03-10 02:34:51,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:34:51,843 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:34:51,845 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69af7f13275c8d3b7939aa59/log.txt.gz 2026-03-10 02:34:52,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:34:52,416 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:34:52,418 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69af7f13275c8d3b7939aa59/lava-logs.yaml 2026-03-10 02:34:52,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:34:52,607 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:34:52,608 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69af7f13275c8d3b7939aa59/lava_callback.json.gz DEBUG: Setup failed for 69af7f13275c8d3b7939aa59. Transitting job node result: incomplete -> fail 2026-03-10 02:34:52,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:34:52,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af7f13275c8d3b7939aa59 HTTP/1.1" 200 None 2026-03-10 02:34:52,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:34:52,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af632c275c8d3b79399db4 HTTP/1.1" 200 None 2026-03-10 02:34:52,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:34:52,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af7f13275c8d3b7939aa59 HTTP/1.1" 200 None 2026-03-10 02:34:52,906 - lava_callback - INFO - Completed processing callback for node 69af7f13275c8d3b7939aa59 2026-03-10 02:34:53,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:34:53,105 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:38258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59684 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48304 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:36:09,112 - lava_callback - INFO - Received callback for node 69af8331275c8d3b7939ab07 INFO: 172.20.0.1:48314 - "POST /node/69af8331275c8d3b7939ab07 HTTP/1.0" 202 Accepted 2026-03-10 02:36:09,118 - lava_callback - INFO - Starting async job submit for node 69af8331275c8d3b7939ab07 2026-03-10 02:36:09,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:36:09,164 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af8331275c8d3b7939ab07 HTTP/1.1" 200 None 2026-03-10 02:36:09,165 - lava_callback - DEBUG - Processing node 69af8331275c8d3b7939ab07: baseline-nfs-arm64 2026-03-10 02:36:09,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:36:09,715 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:36:09,716 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af8331275c8d3b7939ab07/log.txt.gz 2026-03-10 02:36:09,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:36:10,178 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:36:10,180 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af8331275c8d3b7939ab07/lava-logs.yaml 2026-03-10 02:36:10,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:36:10,343 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:36:10,344 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af8331275c8d3b7939ab07/lava_callback.json.gz DEBUG: Setup failed for 69af8331275c8d3b7939ab07. Transitting job node result: incomplete -> fail 2026-03-10 02:36:10,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:36:10,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af8331275c8d3b7939ab07 HTTP/1.1" 200 None 2026-03-10 02:36:10,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:36:10,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 02:36:10,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:36:10,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af8331275c8d3b7939ab07 HTTP/1.1" 200 None 2026-03-10 02:36:10,560 - lava_callback - INFO - Completed processing callback for node 69af8331275c8d3b7939ab07 INFO: 172.20.0.1:48660 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48674 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:36:23,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:36:23,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:37974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35032 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57968 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46146 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:39:02,889 - lava_callback - INFO - Received callback for node 69af83d3275c8d3b7939ab25 INFO: 172.20.0.1:48502 - "POST /node/69af83d3275c8d3b7939ab25 HTTP/1.0" 202 Accepted 2026-03-10 02:39:02,895 - lava_callback - INFO - Starting async job submit for node 69af83d3275c8d3b7939ab25 2026-03-10 02:39:02,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:39:02,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af83d3275c8d3b7939ab25 HTTP/1.1" 200 None 2026-03-10 02:39:02,946 - lava_callback - DEBUG - Processing node 69af83d3275c8d3b7939ab25: baseline-nfs-arm64 2026-03-10 02:39:03,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:39:03,430 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:39:03,432 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af83d3275c8d3b7939ab25/log.txt.gz 2026-03-10 02:39:03,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:39:03,877 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:39:03,879 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af83d3275c8d3b7939ab25/lava-logs.yaml 2026-03-10 02:39:03,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:39:04,027 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:39:04,028 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af83d3275c8d3b7939ab25/lava_callback.json.gz DEBUG: Setup failed for 69af83d3275c8d3b7939ab25. Transitting job node result: incomplete -> fail 2026-03-10 02:39:04,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:39:04,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af83d3275c8d3b7939ab25 HTTP/1.1" 200 None 2026-03-10 02:39:04,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:39:04,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 02:39:04,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:39:04,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af83d3275c8d3b7939ab25 HTTP/1.1" 200 None 2026-03-10 02:39:04,240 - lava_callback - INFO - Completed processing callback for node 69af83d3275c8d3b7939ab25 INFO: 172.20.0.1:48504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33388 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33404 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:39:23,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:39:23,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:40174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41564 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52612 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:41:13,044 - lava_callback - INFO - Received callback for node 69af8476275c8d3b7939ab3d INFO: 172.20.0.1:46926 - "POST /node/69af8476275c8d3b7939ab3d HTTP/1.0" 202 Accepted 2026-03-10 02:41:13,050 - lava_callback - INFO - Starting async job submit for node 69af8476275c8d3b7939ab3d 2026-03-10 02:41:13,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:41:13,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af8476275c8d3b7939ab3d HTTP/1.1" 200 None 2026-03-10 02:41:13,097 - lava_callback - DEBUG - Processing node 69af8476275c8d3b7939ab3d: baseline-arm64-foundriesio 2026-03-10 02:41:13,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:41:13,471 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:41:13,473 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69af8476275c8d3b7939ab3d/log.txt.gz 2026-03-10 02:41:13,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:41:13,819 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:41:13,821 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69af8476275c8d3b7939ab3d/lava-logs.yaml 2026-03-10 02:41:13,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:41:14,003 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:41:14,005 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69af8476275c8d3b7939ab3d/lava_callback.json.gz DEBUG: Setup failed for 69af8476275c8d3b7939ab3d. Transitting job node result: incomplete -> fail 2026-03-10 02:41:14,076 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:41:14,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af8476275c8d3b7939ab3d HTTP/1.1" 200 None 2026-03-10 02:41:14,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:41:14,110 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 02:41:14,112 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:41:14,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af8476275c8d3b7939ab3d HTTP/1.1" 200 None 2026-03-10 02:41:14,213 - lava_callback - INFO - Completed processing callback for node 69af8476275c8d3b7939ab3d INFO: 172.20.0.1:46942 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46956 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:41:23,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:41:23,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:57684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58614 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57602 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37444 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:43:46,796 - lava_callback - INFO - Received callback for node 69af854c275c8d3b7939ab60 INFO: 172.20.0.1:37452 - "POST /node/69af854c275c8d3b7939ab60 HTTP/1.0" 202 Accepted 2026-03-10 02:43:46,802 - lava_callback - INFO - Starting async job submit for node 69af854c275c8d3b7939ab60 2026-03-10 02:43:46,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:43:46,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af854c275c8d3b7939ab60 HTTP/1.1" 200 None 2026-03-10 02:43:46,821 - lava_callback - DEBUG - Processing node 69af854c275c8d3b7939ab60: baseline-arm64 2026-03-10 02:43:46,825 - lava_callback - WARNING - Failed to upload log 2026-03-10 02:43:46,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:43:46,978 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:43:46,979 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af854c275c8d3b7939ab60/lava-logs.yaml 2026-03-10 02:43:46,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:43:47,127 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:43:47,129 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af854c275c8d3b7939ab60/lava_callback.json.gz 2026-03-10 02:43:47,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:43:47,146 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af854c275c8d3b7939ab60 HTTP/1.1" 200 None 2026-03-10 02:43:47,147 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:43:47,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 02:43:47,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:43:47,183 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af854c275c8d3b7939ab60 HTTP/1.1" 200 None 2026-03-10 02:43:47,188 - lava_callback - INFO - Completed processing callback for node 69af854c275c8d3b7939ab60 2026-03-10 02:43:53,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:43:53,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:39994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56306 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34264 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:44:44,394 - lava_callback - INFO - Received callback for node 69af802c275c8d3b7939aa89 INFO: 172.20.0.1:37062 - "POST /node/69af802c275c8d3b7939aa89 HTTP/1.0" 202 Accepted 2026-03-10 02:44:44,400 - lava_callback - INFO - Starting async job submit for node 69af802c275c8d3b7939aa89 2026-03-10 02:44:44,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:44:44,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af802c275c8d3b7939aa89 HTTP/1.1" 200 None 2026-03-10 02:44:44,445 - lava_callback - DEBUG - Processing node 69af802c275c8d3b7939aa89: baseline-arm64-broonie 2026-03-10 02:44:44,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:44:45,036 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:44:45,038 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af802c275c8d3b7939aa89/log.txt.gz 2026-03-10 02:44:45,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:44:45,554 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:44:45,556 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af802c275c8d3b7939aa89/lava-logs.yaml 2026-03-10 02:44:45,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:44:45,725 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:44:45,727 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af802c275c8d3b7939aa89/lava_callback.json.gz 2026-03-10 02:44:45,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:44:45,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af802c275c8d3b7939aa89 HTTP/1.1" 200 None 2026-03-10 02:44:45,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:44:45,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 02:44:45,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:44:45,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af802c275c8d3b7939aa89 HTTP/1.1" 200 None 2026-03-10 02:44:45,885 - lava_callback - INFO - Completed processing callback for node 69af802c275c8d3b7939aa89 INFO: 172.20.0.1:37072 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:44:53,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:44:53,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:58750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48006 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37436 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35196 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:47:11,213 - lava_callback - INFO - Received callback for node 69af6798275c8d3b79399e45 INFO: 172.20.0.1:59868 - "POST /node/69af6798275c8d3b79399e45 HTTP/1.0" 202 Accepted 2026-03-10 02:47:11,219 - lava_callback - INFO - Starting async job submit for node 69af6798275c8d3b79399e45 2026-03-10 02:47:11,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:47:11,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6798275c8d3b79399e45 HTTP/1.1" 200 None 2026-03-10 02:47:11,261 - lava_callback - DEBUG - Processing node 69af6798275c8d3b79399e45: baseline-arm-broonie 2026-03-10 02:47:11,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:47:11,626 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:47:11,627 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69af6798275c8d3b79399e45/log.txt.gz 2026-03-10 02:47:11,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:47:11,947 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:47:11,949 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69af6798275c8d3b79399e45/lava-logs.yaml 2026-03-10 02:47:11,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:47:12,115 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:47:12,117 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69af6798275c8d3b79399e45/lava_callback.json.gz 2026-03-10 02:47:12,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:47:12,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6798275c8d3b79399e45 HTTP/1.1" 200 None 2026-03-10 02:47:12,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:47:12,190 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af632c275c8d3b79399db4 HTTP/1.1" 200 None 2026-03-10 02:47:12,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:47:12,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af6798275c8d3b79399e45 HTTP/1.1" 200 None 2026-03-10 02:47:12,243 - lava_callback - INFO - Completed processing callback for node 69af6798275c8d3b79399e45 INFO: 172.20.0.1:59872 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59878 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:47:23,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:47:23,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:39842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46958 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:47:41,980 - lava_callback - INFO - Received callback for node 69af7575275c8d3b7939a878 INFO: 172.20.0.1:47750 - "POST /node/69af7575275c8d3b7939a878 HTTP/1.0" 202 Accepted 2026-03-10 02:47:41,987 - lava_callback - INFO - Starting async job submit for node 69af7575275c8d3b7939a878 2026-03-10 02:47:42,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:47:42,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af7575275c8d3b7939a878 HTTP/1.1" 200 None 2026-03-10 02:47:42,033 - lava_callback - DEBUG - Processing node 69af7575275c8d3b7939a878: baseline-arm64 2026-03-10 02:47:42,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:47:42,569 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:47:42,571 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af7575275c8d3b7939a878/log.txt.gz 2026-03-10 02:47:42,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:47:43,011 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:47:43,013 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af7575275c8d3b7939a878/lava-logs.yaml 2026-03-10 02:47:43,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:47:43,149 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:47:43,151 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af7575275c8d3b7939a878/lava_callback.json.gz DEBUG: Setup failed for 69af7575275c8d3b7939a878. Transitting job node result: incomplete -> fail 2026-03-10 02:47:43,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:47:43,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af7575275c8d3b7939a878 HTTP/1.1" 200 None 2026-03-10 02:47:43,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:47:43,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 02:47:43,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:47:43,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af7575275c8d3b7939a878 HTTP/1.1" 200 None 2026-03-10 02:47:43,363 - lava_callback - INFO - Completed processing callback for node 69af7575275c8d3b7939a878 INFO: 172.20.0.1:47766 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:47:50,002 - lava_callback - INFO - Received callback for node 69af7bf8275c8d3b7939a9aa INFO: 172.20.0.1:49784 - "POST /node/69af7bf8275c8d3b7939a9aa HTTP/1.0" 202 Accepted 2026-03-10 02:47:50,010 - lava_callback - INFO - Starting async job submit for node 69af7bf8275c8d3b7939a9aa 2026-03-10 02:47:50,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:47:50,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af7bf8275c8d3b7939a9aa HTTP/1.1" 200 None 2026-03-10 02:47:50,029 - lava_callback - DEBUG - Processing node 69af7bf8275c8d3b7939a9aa: baseline-arm64 2026-03-10 02:47:50,035 - lava_callback - WARNING - Failed to upload log 2026-03-10 02:47:50,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:47:50,161 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:47:50,163 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af7bf8275c8d3b7939a9aa/lava-logs.yaml 2026-03-10 02:47:50,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:47:50,275 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:47:50,277 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af7bf8275c8d3b7939a9aa/lava_callback.json.gz 2026-03-10 02:47:50,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:47:50,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af7bf8275c8d3b7939a9aa HTTP/1.1" 200 None 2026-03-10 02:47:50,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:47:50,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 02:47:50,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:47:50,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af7bf8275c8d3b7939a9aa HTTP/1.1" 200 None 2026-03-10 02:47:50,344 - lava_callback - INFO - Completed processing callback for node 69af7bf8275c8d3b7939a9aa 2026-03-10 02:47:53,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:47:53,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-10 02:47:56,297 - lava_callback - INFO - Received callback for node 69af7620275c8d3b7939a890 INFO: 172.20.0.1:49800 - "POST /node/69af7620275c8d3b7939a890 HTTP/1.0" 202 Accepted 2026-03-10 02:47:56,303 - lava_callback - INFO - Starting async job submit for node 69af7620275c8d3b7939a890 2026-03-10 02:47:56,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:47:56,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af7620275c8d3b7939a890 HTTP/1.1" 200 None 2026-03-10 02:47:56,354 - lava_callback - DEBUG - Processing node 69af7620275c8d3b7939a890: baseline-nfs-arm64 INFO: 172.20.0.1:49808 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:47:56,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:47:56,858 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:47:56,859 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af7620275c8d3b7939a890/log.txt.gz 2026-03-10 02:47:57,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:47:57,296 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:47:57,298 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af7620275c8d3b7939a890/lava-logs.yaml 2026-03-10 02:47:57,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:47:57,433 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:47:57,435 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af7620275c8d3b7939a890/lava_callback.json.gz DEBUG: Setup failed for 69af7620275c8d3b7939a890. Transitting job node result: incomplete -> fail 2026-03-10 02:47:57,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:47:57,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af7620275c8d3b7939a890 HTTP/1.1" 200 None 2026-03-10 02:47:57,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:47:57,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 02:47:57,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:47:57,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af7620275c8d3b7939a890 HTTP/1.1" 200 None 2026-03-10 02:47:57,654 - lava_callback - INFO - Completed processing callback for node 69af7620275c8d3b7939a890 INFO: 172.20.0.1:45536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45552 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:48:10,663 - lava_callback - INFO - Received callback for node 69af7d91275c8d3b7939a9ff INFO: 172.20.0.1:36122 - "POST /node/69af7d91275c8d3b7939a9ff HTTP/1.0" 202 Accepted 2026-03-10 02:48:10,669 - lava_callback - INFO - Starting async job submit for node 69af7d91275c8d3b7939a9ff 2026-03-10 02:48:10,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:48:10,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af7d91275c8d3b7939a9ff HTTP/1.1" 200 None 2026-03-10 02:48:10,688 - lava_callback - DEBUG - Processing node 69af7d91275c8d3b7939a9ff: baseline-nfs-arm64 2026-03-10 02:48:10,694 - lava_callback - WARNING - Failed to upload log 2026-03-10 02:48:10,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:48:10,813 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:48:10,815 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af7d91275c8d3b7939a9ff/lava-logs.yaml 2026-03-10 02:48:10,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:48:10,929 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:48:10,930 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af7d91275c8d3b7939a9ff/lava_callback.json.gz 2026-03-10 02:48:10,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:48:10,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af7d91275c8d3b7939a9ff HTTP/1.1" 200 None 2026-03-10 02:48:10,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:48:10,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 02:48:10,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:48:10,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af7d91275c8d3b7939a9ff HTTP/1.1" 200 None 2026-03-10 02:48:10,997 - lava_callback - INFO - Completed processing callback for node 69af7d91275c8d3b7939a9ff INFO: 172.20.0.1:36126 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36136 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:48:23,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:48:23,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:35114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51932 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:48:51,964 - lava_callback - INFO - Received callback for node 69af8630275c8d3b7939ab8c INFO: 172.20.0.1:47822 - "POST /node/69af8630275c8d3b7939ab8c HTTP/1.0" 202 Accepted 2026-03-10 02:48:51,971 - lava_callback - INFO - Starting async job submit for node 69af8630275c8d3b7939ab8c 2026-03-10 02:48:52,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:48:52,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af8630275c8d3b7939ab8c HTTP/1.1" 200 None 2026-03-10 02:48:52,024 - lava_callback - DEBUG - Processing node 69af8630275c8d3b7939ab8c: baseline-arm64 2026-03-10 02:48:52,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:48:52,713 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:48:52,714 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af8630275c8d3b7939ab8c/log.txt.gz 2026-03-10 02:48:53,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:48:53,277 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:48:53,279 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af8630275c8d3b7939ab8c/lava-logs.yaml 2026-03-10 02:48:53,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:48:53,480 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:48:53,481 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af8630275c8d3b7939ab8c/lava_callback.json.gz DEBUG: Setup failed for 69af8630275c8d3b7939ab8c. Transitting job node result: incomplete -> fail 2026-03-10 02:48:53,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:48:53,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af8630275c8d3b7939ab8c HTTP/1.1" 200 None 2026-03-10 02:48:53,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:48:53,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 02:48:53,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:48:53,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af8630275c8d3b7939ab8c HTTP/1.1" 200 None 2026-03-10 02:48:53,689 - lava_callback - INFO - Completed processing callback for node 69af8630275c8d3b7939ab8c INFO: 172.20.0.1:47830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37642 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37650 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:49:23,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:49:23,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:43546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52310 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40472 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:50:28,426 - lava_callback - INFO - Received callback for node 69af8686275c8d3b7939abb0 INFO: 172.20.0.1:40486 - "POST /node/69af8686275c8d3b7939abb0 HTTP/1.0" 202 Accepted 2026-03-10 02:50:28,432 - lava_callback - INFO - Starting async job submit for node 69af8686275c8d3b7939abb0 2026-03-10 02:50:28,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:50:28,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af8686275c8d3b7939abb0 HTTP/1.1" 200 None 2026-03-10 02:50:28,480 - lava_callback - DEBUG - Processing node 69af8686275c8d3b7939abb0: baseline-nfs-arm64 2026-03-10 02:50:28,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:50:28,940 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:50:28,941 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af8686275c8d3b7939abb0/log.txt.gz 2026-03-10 02:50:29,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:50:29,343 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:50:29,345 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af8686275c8d3b7939abb0/lava-logs.yaml 2026-03-10 02:50:29,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:50:29,509 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:50:29,510 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af8686275c8d3b7939abb0/lava_callback.json.gz DEBUG: Setup failed for 69af8686275c8d3b7939abb0. Transitting job node result: incomplete -> fail 2026-03-10 02:50:29,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:50:29,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af8686275c8d3b7939abb0 HTTP/1.1" 200 None 2026-03-10 02:50:29,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:50:29,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 02:50:29,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:50:29,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af8686275c8d3b7939abb0 HTTP/1.1" 200 None 2026-03-10 02:50:29,713 - lava_callback - INFO - Completed processing callback for node 69af8686275c8d3b7939abb0 INFO: 172.20.0.1:47112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35996 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:50:51,632 - lava_callback - INFO - Received callback for node 69af67c3275c8d3b79399ed8 INFO: 172.20.0.1:57912 - "POST /node/69af67c3275c8d3b79399ed8 HTTP/1.0" 202 Accepted 2026-03-10 02:50:51,642 - lava_callback - INFO - Starting async job submit for node 69af67c3275c8d3b79399ed8 2026-03-10 02:50:51,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:50:51,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af67c3275c8d3b79399ed8 HTTP/1.1" 200 None 2026-03-10 02:50:51,683 - lava_callback - DEBUG - Processing node 69af67c3275c8d3b79399ed8: ltp-cap_bounds 2026-03-10 02:50:52,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:50:52,253 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:50:52,255 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69af67c3275c8d3b79399ed8/log.txt.gz 2026-03-10 02:50:52,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:50:52,678 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:50:52,680 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69af67c3275c8d3b79399ed8/lava-logs.yaml 2026-03-10 02:50:52,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:50:52,834 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:50:52,835 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69af67c3275c8d3b79399ed8/lava_callback.json.gz 2026-03-10 02:50:52,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:50:52,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af67c3275c8d3b79399ed8 HTTP/1.1" 200 None 2026-03-10 02:50:52,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:50:52,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af632c275c8d3b79399db4 HTTP/1.1" 200 None 2026-03-10 02:50:52,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:50:52,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af67c3275c8d3b79399ed8 HTTP/1.1" 200 None 2026-03-10 02:50:52,918 - lava_callback - INFO - Completed processing callback for node 69af67c3275c8d3b79399ed8 2026-03-10 02:50:53,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:50:53,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:57918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36042 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:51:11,750 - lava_callback - INFO - Received callback for node 69af67c4275c8d3b79399edb INFO: 172.20.0.1:48268 - "POST /node/69af67c4275c8d3b79399edb HTTP/1.0" 202 Accepted 2026-03-10 02:51:11,757 - lava_callback - INFO - Starting async job submit for node 69af67c4275c8d3b79399edb 2026-03-10 02:51:11,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:51:11,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af67c4275c8d3b79399edb HTTP/1.1" 200 None 2026-03-10 02:51:11,798 - lava_callback - DEBUG - Processing node 69af67c4275c8d3b79399edb: ltp-containers 2026-03-10 02:51:12,163 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:51:12,313 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:51:12,315 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-containers-69af67c4275c8d3b79399edb/log.txt.gz 2026-03-10 02:51:12,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:51:12,715 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:51:12,717 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-containers-69af67c4275c8d3b79399edb/lava-logs.yaml 2026-03-10 02:51:12,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:51:12,895 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:51:12,897 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-containers-69af67c4275c8d3b79399edb/lava_callback.json.gz 2026-03-10 02:51:12,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:51:12,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af67c4275c8d3b79399edb HTTP/1.1" 200 None 2026-03-10 02:51:12,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:51:12,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af632c275c8d3b79399db4 HTTP/1.1" 200 None 2026-03-10 02:51:12,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:51:13,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af67c4275c8d3b79399edb HTTP/1.1" 200 None 2026-03-10 02:51:13,007 - lava_callback - INFO - Completed processing callback for node 69af67c4275c8d3b79399edb INFO: 172.20.0.1:48278 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48294 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:51:19,471 - lava_callback - INFO - Received callback for node 69af871e275c8d3b7939abdd INFO: 172.20.0.1:48300 - "POST /node/69af871e275c8d3b7939abdd HTTP/1.0" 202 Accepted 2026-03-10 02:51:19,476 - lava_callback - INFO - Starting async job submit for node 69af871e275c8d3b7939abdd 2026-03-10 02:51:19,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:51:19,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af871e275c8d3b7939abdd HTTP/1.1" 200 None 2026-03-10 02:51:19,495 - lava_callback - DEBUG - Processing node 69af871e275c8d3b7939abdd: baseline-arm64 2026-03-10 02:51:19,499 - lava_callback - WARNING - Failed to upload log 2026-03-10 02:51:19,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:51:19,626 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:51:19,628 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af871e275c8d3b7939abdd/lava-logs.yaml 2026-03-10 02:51:19,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:51:19,739 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:51:19,741 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af871e275c8d3b7939abdd/lava_callback.json.gz 2026-03-10 02:51:19,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:51:19,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af871e275c8d3b7939abdd HTTP/1.1" 200 None 2026-03-10 02:51:19,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:51:19,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 02:51:19,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:51:19,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af871e275c8d3b7939abdd HTTP/1.1" 200 None 2026-03-10 02:51:19,807 - lava_callback - INFO - Completed processing callback for node 69af871e275c8d3b7939abdd 2026-03-10 02:51:23,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:51:23,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:41490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53900 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:51:51,552 - lava_callback - INFO - Received callback for node 69af86d3275c8d3b7939abc4 INFO: 172.20.0.1:42052 - "POST /node/69af86d3275c8d3b7939abc4 HTTP/1.0" 202 Accepted 2026-03-10 02:51:51,558 - lava_callback - INFO - Starting async job submit for node 69af86d3275c8d3b7939abc4 2026-03-10 02:51:51,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:51:51,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af86d3275c8d3b7939abc4 HTTP/1.1" 200 None 2026-03-10 02:51:51,607 - lava_callback - DEBUG - Processing node 69af86d3275c8d3b7939abc4: baseline-nfs-arm64 2026-03-10 02:51:52,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:51:52,279 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:51:52,280 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af86d3275c8d3b7939abc4/log.txt.gz 2026-03-10 02:51:52,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:51:52,804 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:51:52,806 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af86d3275c8d3b7939abc4/lava-logs.yaml 2026-03-10 02:51:52,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:51:52,999 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:51:53,001 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af86d3275c8d3b7939abc4/lava_callback.json.gz DEBUG: Setup failed for 69af86d3275c8d3b7939abc4. Transitting job node result: incomplete -> fail 2026-03-10 02:51:53,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:51:53,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af86d3275c8d3b7939abc4 HTTP/1.1" 200 None 2026-03-10 02:51:53,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:51:53,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 02:51:53,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:51:53,157 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af86d3275c8d3b7939abc4 HTTP/1.1" 200 None 2026-03-10 02:51:53,244 - lava_callback - INFO - Completed processing callback for node 69af86d3275c8d3b7939abc4 2026-03-10 02:51:53,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:51:53,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:42060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57976 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57992 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:52:25,186 - lava_callback - INFO - Received callback for node 69af67c6275c8d3b79399ede INFO: 172.20.0.1:55408 - "POST /node/69af67c6275c8d3b79399ede HTTP/1.0" 202 Accepted 2026-03-10 02:52:25,194 - lava_callback - INFO - Starting async job submit for node 69af67c6275c8d3b79399ede 2026-03-10 02:52:25,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:52:25,236 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af67c6275c8d3b79399ede HTTP/1.1" 200 None 2026-03-10 02:52:25,236 - lava_callback - DEBUG - Processing node 69af67c6275c8d3b79399ede: ltp-controllers 2026-03-10 02:52:25,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:52:25,828 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:52:25,830 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-controllers-69af67c6275c8d3b79399ede/log.txt.gz 2026-03-10 02:52:26,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:52:26,308 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:52:26,310 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-controllers-69af67c6275c8d3b79399ede/lava-logs.yaml 2026-03-10 02:52:26,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:52:26,465 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:52:26,467 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-controllers-69af67c6275c8d3b79399ede/lava_callback.json.gz 2026-03-10 02:52:26,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:52:26,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af67c6275c8d3b79399ede HTTP/1.1" 200 None 2026-03-10 02:52:26,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:55422 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:52:26,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af632c275c8d3b79399db4 HTTP/1.1" 200 None 2026-03-10 02:52:26,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:52:26,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af67c6275c8d3b79399ede HTTP/1.1" 200 None 2026-03-10 02:52:26,550 - lava_callback - INFO - Completed processing callback for node 69af67c6275c8d3b79399ede INFO: 172.20.0.1:49596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56234 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:52:53,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:52:53,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:46172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37556 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55518 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55528 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:54:17,421 - lava_callback - INFO - Received callback for node 69ace4c6bcd207e915d78050 INFO: 172.20.0.1:55530 - "POST /node/69ace4c6bcd207e915d78050 HTTP/1.0" 202 Accepted 2026-03-10 02:54:17,426 - lava_callback - INFO - Starting async job submit for node 69ace4c6bcd207e915d78050 2026-03-10 02:54:17,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:54:17,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ace4c6bcd207e915d78050 HTTP/1.1" 200 None 2026-03-10 02:54:17,450 - lava_callback - DEBUG - Processing node 69ace4c6bcd207e915d78050: baseline-arm-baylibre 2026-03-10 02:54:17,450 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ace4c6bcd207e915d78050 2026-03-10 02:54:17,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:54:17,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ace4c6bcd207e915d78050 HTTP/1.1" 200 None 2026-03-10 02:54:17,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:54:17,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abdf2740deb775998f3ba4 HTTP/1.1" 200 None 2026-03-10 02:54:17,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:54:17,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ace4c6bcd207e915d78050 HTTP/1.1" 200 None 2026-03-10 02:54:17,512 - lava_callback - ERROR - Error processing callback for node 69ace4c6bcd207e915d78050: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34068 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:55:06,430 - lava_callback - INFO - Received callback for node 69af87b4275c8d3b7939ac08 INFO: 172.20.0.1:34076 - "POST /node/69af87b4275c8d3b7939ac08 HTTP/1.0" 202 Accepted 2026-03-10 02:55:06,436 - lava_callback - INFO - Starting async job submit for node 69af87b4275c8d3b7939ac08 2026-03-10 02:55:06,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:55:06,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af87b4275c8d3b7939ac08 HTTP/1.1" 200 None 2026-03-10 02:55:06,497 - lava_callback - DEBUG - Processing node 69af87b4275c8d3b7939ac08: baseline-arm64 INFO: 172.20.0.1:34082 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:55:06,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:55:07,060 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:55:07,062 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af87b4275c8d3b7939ac08/log.txt.gz 2026-03-10 02:55:07,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:55:07,491 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:55:07,493 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af87b4275c8d3b7939ac08/lava-logs.yaml 2026-03-10 02:55:07,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:55:07,671 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:55:07,673 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af87b4275c8d3b7939ac08/lava_callback.json.gz DEBUG: Setup failed for 69af87b4275c8d3b7939ac08. Transitting job node result: incomplete -> fail 2026-03-10 02:55:07,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:55:07,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af87b4275c8d3b7939ac08 HTTP/1.1" 200 None 2026-03-10 02:55:07,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:55:07,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 02:55:07,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:55:07,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af87b4275c8d3b7939ac08 HTTP/1.1" 200 None 2026-03-10 02:55:07,872 - lava_callback - INFO - Completed processing callback for node 69af87b4275c8d3b7939ac08 2026-03-10 02:55:11,807 - lava_callback - INFO - Received callback for node 69af67cc275c8d3b79399ee8 INFO: 172.20.0.1:45932 - "POST /node/69af67cc275c8d3b79399ee8 HTTP/1.0" 202 Accepted 2026-03-10 02:55:11,813 - lava_callback - INFO - Starting async job submit for node 69af67cc275c8d3b79399ee8 2026-03-10 02:55:11,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:55:11,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af67cc275c8d3b79399ee8 HTTP/1.1" 200 None 2026-03-10 02:55:11,879 - lava_callback - DEBUG - Processing node 69af67cc275c8d3b79399ee8: ltp-fs_bind 2026-03-10 02:55:12,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:55:12,415 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:55:12,416 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69af67cc275c8d3b79399ee8/log.txt.gz 2026-03-10 02:55:12,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:55:12,859 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:55:12,861 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69af67cc275c8d3b79399ee8/lava-logs.yaml 2026-03-10 02:55:12,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:55:13,035 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:55:13,037 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69af67cc275c8d3b79399ee8/lava_callback.json.gz 2026-03-10 02:55:13,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:55:13,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af67cc275c8d3b79399ee8 HTTP/1.1" 200 None 2026-03-10 02:55:13,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:55:13,093 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af632c275c8d3b79399db4 HTTP/1.1" 200 None 2026-03-10 02:55:13,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:55:13,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af67cc275c8d3b79399ee8 HTTP/1.1" 200 None 2026-03-10 02:55:13,124 - lava_callback - INFO - Completed processing callback for node 69af67cc275c8d3b79399ee8 INFO: 172.20.0.1:45936 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45940 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:55:17,272 - lava_callback - INFO - Received callback for node 69ace502bcd207e915d78062 INFO: 172.20.0.1:45946 - "POST /node/69ace502bcd207e915d78062 HTTP/1.0" 202 Accepted 2026-03-10 02:55:17,277 - lava_callback - INFO - Starting async job submit for node 69ace502bcd207e915d78062 2026-03-10 02:55:17,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:55:17,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ace502bcd207e915d78062 HTTP/1.1" 200 None 2026-03-10 02:55:17,294 - lava_callback - DEBUG - Processing node 69ace502bcd207e915d78062: baseline-arm-baylibre 2026-03-10 02:55:17,294 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ace502bcd207e915d78062 2026-03-10 02:55:17,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:55:17,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ace502bcd207e915d78062 HTTP/1.1" 200 None 2026-03-10 02:55:17,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:55:17,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abdf2740deb775998f3ba4 HTTP/1.1" 200 None 2026-03-10 02:55:17,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:55:17,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ace502bcd207e915d78062 HTTP/1.1" 200 None 2026-03-10 02:55:17,350 - lava_callback - ERROR - Error processing callback for node 69ace502bcd207e915d78062: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-10 02:55:23,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:55:23,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:47572 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:55:31,525 - lava_callback - INFO - Received callback for node 69af67c8275c8d3b79399ee2 INFO: 172.20.0.1:57642 - "POST /node/69af67c8275c8d3b79399ee2 HTTP/1.0" 202 Accepted 2026-03-10 02:55:31,531 - lava_callback - INFO - Starting async job submit for node 69af67c8275c8d3b79399ee2 2026-03-10 02:55:31,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:55:31,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af67c8275c8d3b79399ee2 HTTP/1.1" 200 None 2026-03-10 02:55:31,573 - lava_callback - DEBUG - Processing node 69af67c8275c8d3b79399ee2: ltp-cve 2026-03-10 02:55:31,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:55:32,112 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:55:32,114 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cve-69af67c8275c8d3b79399ee2/log.txt.gz 2026-03-10 02:55:32,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:55:32,539 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:55:32,542 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cve-69af67c8275c8d3b79399ee2/lava-logs.yaml 2026-03-10 02:55:32,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:55:32,698 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:55:32,700 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cve-69af67c8275c8d3b79399ee2/lava_callback.json.gz 2026-03-10 02:55:32,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:55:32,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af67c8275c8d3b79399ee2 HTTP/1.1" 200 None 2026-03-10 02:55:32,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:55:32,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af632c275c8d3b79399db4 HTTP/1.1" 200 None 2026-03-10 02:55:32,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:55:32,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af67c8275c8d3b79399ee2 HTTP/1.1" 200 None 2026-03-10 02:55:32,799 - lava_callback - INFO - Completed processing callback for node 69af67c8275c8d3b79399ee2 INFO: 172.20.0.1:57650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33432 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:55:51,581 - lava_callback - INFO - Received callback for node 69af67ce275c8d3b79399eeb INFO: 172.20.0.1:59108 - "POST /node/69af67ce275c8d3b79399eeb HTTP/1.0" 202 Accepted 2026-03-10 02:55:51,588 - lava_callback - INFO - Starting async job submit for node 69af67ce275c8d3b79399eeb 2026-03-10 02:55:51,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:55:51,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af67ce275c8d3b79399eeb HTTP/1.1" 200 None 2026-03-10 02:55:51,629 - lava_callback - DEBUG - Processing node 69af67ce275c8d3b79399eeb: ltp-fsx 2026-03-10 02:55:51,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:55:52,106 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:55:52,108 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fsx-69af67ce275c8d3b79399eeb/log.txt.gz 2026-03-10 02:55:52,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:55:52,506 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:55:52,508 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fsx-69af67ce275c8d3b79399eeb/lava-logs.yaml 2026-03-10 02:55:52,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:55:52,667 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:55:52,668 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fsx-69af67ce275c8d3b79399eeb/lava_callback.json.gz 2026-03-10 02:55:52,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:55:52,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af67ce275c8d3b79399eeb HTTP/1.1" 200 None 2026-03-10 02:55:52,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:55:52,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af632c275c8d3b79399db4 HTTP/1.1" 200 None 2026-03-10 02:55:52,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:55:52,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af67ce275c8d3b79399eeb HTTP/1.1" 200 None 2026-03-10 02:55:52,769 - lava_callback - INFO - Completed processing callback for node 69af67ce275c8d3b79399eeb 2026-03-10 02:55:53,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:55:53,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:59114 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:55:57,349 - lava_callback - INFO - Received callback for node 69ace52bbcd207e915d78077 INFO: 172.20.0.1:59126 - "POST /node/69ace52bbcd207e915d78077 HTTP/1.0" 202 Accepted 2026-03-10 02:55:57,354 - lava_callback - INFO - Starting async job submit for node 69ace52bbcd207e915d78077 2026-03-10 02:55:57,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:55:57,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ace52bbcd207e915d78077 HTTP/1.1" 200 None 2026-03-10 02:55:57,374 - lava_callback - DEBUG - Processing node 69ace52bbcd207e915d78077: baseline-arm-baylibre 2026-03-10 02:55:57,375 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ace52bbcd207e915d78077 2026-03-10 02:55:57,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:55:57,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ace52bbcd207e915d78077 HTTP/1.1" 200 None 2026-03-10 02:55:57,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:55:57,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abdf2740deb775998f3ba4 HTTP/1.1" 200 None 2026-03-10 02:55:57,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:55:57,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ace52bbcd207e915d78077 HTTP/1.1" 200 None 2026-03-10 02:55:57,437 - lava_callback - ERROR - Error processing callback for node 69ace52bbcd207e915d78077: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41708 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41716 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:56:24,743 - lava_callback - INFO - Received callback for node 69af67cf275c8d3b79399eee INFO: 172.20.0.1:56990 - "POST /node/69af67cf275c8d3b79399eee HTTP/1.0" 202 Accepted 2026-03-10 02:56:24,750 - lava_callback - INFO - Starting async job submit for node 69af67cf275c8d3b79399eee 2026-03-10 02:56:24,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:56:24,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af67cf275c8d3b79399eee HTTP/1.1" 200 None 2026-03-10 02:56:24,790 - lava_callback - DEBUG - Processing node 69af67cf275c8d3b79399eee: ltp-input 2026-03-10 02:56:25,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:56:25,273 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:56:25,274 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-input-69af67cf275c8d3b79399eee/log.txt.gz 2026-03-10 02:56:25,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:56:25,868 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:56:25,870 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-input-69af67cf275c8d3b79399eee/lava-logs.yaml 2026-03-10 02:56:25,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:56:26,029 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:56:26,030 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-input-69af67cf275c8d3b79399eee/lava_callback.json.gz 2026-03-10 02:56:26,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:56:26,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af67cf275c8d3b79399eee HTTP/1.1" 200 None 2026-03-10 02:56:26,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:56:26,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af632c275c8d3b79399db4 HTTP/1.1" 200 None 2026-03-10 02:56:26,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:56:26,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af67cf275c8d3b79399eee HTTP/1.1" 200 None 2026-03-10 02:56:26,119 - lava_callback - INFO - Completed processing callback for node 69af67cf275c8d3b79399eee INFO: 172.20.0.1:56996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40402 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:56:37,354 - lava_callback - INFO - Received callback for node 69ace552bcd207e915d78090 INFO: 172.20.0.1:40412 - "POST /node/69ace552bcd207e915d78090 HTTP/1.0" 202 Accepted 2026-03-10 02:56:37,360 - lava_callback - INFO - Starting async job submit for node 69ace552bcd207e915d78090 2026-03-10 02:56:37,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:56:37,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ace552bcd207e915d78090 HTTP/1.1" 200 None 2026-03-10 02:56:37,376 - lava_callback - DEBUG - Processing node 69ace552bcd207e915d78090: baseline-arm-baylibre 2026-03-10 02:56:37,376 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ace552bcd207e915d78090 2026-03-10 02:56:37,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:56:37,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ace552bcd207e915d78090 HTTP/1.1" 200 None 2026-03-10 02:56:37,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:56:37,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abdf2740deb775998f3ba4 HTTP/1.1" 200 None 2026-03-10 02:56:37,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:56:37,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ace552bcd207e915d78090 HTTP/1.1" 200 None 2026-03-10 02:56:37,436 - lava_callback - ERROR - Error processing callback for node 69ace552bcd207e915d78090: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59346 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:56:53,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:56:53,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:60994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51412 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51428 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:57:17,433 - lava_callback - INFO - Received callback for node 69ace57cbcd207e915d780a8 INFO: 172.20.0.1:51434 - "POST /node/69ace57cbcd207e915d780a8 HTTP/1.0" 202 Accepted 2026-03-10 02:57:17,439 - lava_callback - INFO - Starting async job submit for node 69ace57cbcd207e915d780a8 2026-03-10 02:57:17,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:57:17,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ace57cbcd207e915d780a8 HTTP/1.1" 200 None 2026-03-10 02:57:17,454 - lava_callback - DEBUG - Processing node 69ace57cbcd207e915d780a8: baseline-arm-baylibre 2026-03-10 02:57:17,454 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ace57cbcd207e915d780a8 2026-03-10 02:57:17,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:57:17,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ace57cbcd207e915d780a8 HTTP/1.1" 200 None 2026-03-10 02:57:17,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:57:17,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abdf2740deb775998f3ba4 HTTP/1.1" 200 None 2026-03-10 02:57:17,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:57:17,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ace57cbcd207e915d780a8 HTTP/1.1" 200 None 2026-03-10 02:57:17,540 - lava_callback - ERROR - Error processing callback for node 69ace57cbcd207e915d780a8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58682 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47034 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:59:11,786 - lava_callback - INFO - Received callback for node 69af67d1275c8d3b79399ef1 INFO: 172.20.0.1:51460 - "POST /node/69af67d1275c8d3b79399ef1 HTTP/1.0" 202 Accepted 2026-03-10 02:59:11,792 - lava_callback - INFO - Starting async job submit for node 69af67d1275c8d3b79399ef1 2026-03-10 02:59:11,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:59:11,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af67d1275c8d3b79399ef1 HTTP/1.1" 200 None 2026-03-10 02:59:11,843 - lava_callback - DEBUG - Processing node 69af67d1275c8d3b79399ef1: ltp-ipc 2026-03-10 02:59:12,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:59:12,344 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:59:12,345 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-ipc-69af67d1275c8d3b79399ef1/log.txt.gz 2026-03-10 02:59:12,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:59:12,777 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:59:12,779 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-ipc-69af67d1275c8d3b79399ef1/lava-logs.yaml 2026-03-10 02:59:12,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:59:12,985 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:59:12,987 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-ipc-69af67d1275c8d3b79399ef1/lava_callback.json.gz 2026-03-10 02:59:13,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:59:13,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af67d1275c8d3b79399ef1 HTTP/1.1" 200 None 2026-03-10 02:59:13,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:59:13,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af632c275c8d3b79399db4 HTTP/1.1" 200 None 2026-03-10 02:59:13,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:59:13,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af67d1275c8d3b79399ef1 HTTP/1.1" 200 None 2026-03-10 02:59:13,093 - lava_callback - INFO - Completed processing callback for node 69af67d1275c8d3b79399ef1 INFO: 172.20.0.1:51476 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51486 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:59:23,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:59:23,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:36558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48868 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:59:44,624 - lava_callback - INFO - Received callback for node 69ace60abcd207e915d780f7 INFO: 172.20.0.1:52664 - "POST /node/69ace60abcd207e915d780f7 HTTP/1.0" 202 Accepted 2026-03-10 02:59:44,630 - lava_callback - INFO - Starting async job submit for node 69ace60abcd207e915d780f7 2026-03-10 02:59:44,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:59:44,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ace60abcd207e915d780f7 HTTP/1.1" 200 None 2026-03-10 02:59:44,646 - lava_callback - DEBUG - Processing node 69ace60abcd207e915d780f7: baseline-arm-clabbe 2026-03-10 02:59:44,647 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ace60abcd207e915d780f7 2026-03-10 02:59:44,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:59:44,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ace60abcd207e915d780f7 HTTP/1.1" 200 None 2026-03-10 02:59:44,665 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:59:44,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abdf2740deb775998f3ba4 HTTP/1.1" 200 None 2026-03-10 02:59:44,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:59:44,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ace60abcd207e915d780f7 HTTP/1.1" 200 None 2026-03-10 02:59:44,710 - lava_callback - ERROR - Error processing callback for node 69ace60abcd207e915d780f7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52678 - "GET / HTTP/1.0" 200 OK 2026-03-10 02:59:51,906 - lava_callback - INFO - Received callback for node 69af67d2275c8d3b79399ef4 INFO: 172.20.0.1:57820 - "POST /node/69af67d2275c8d3b79399ef4 HTTP/1.0" 202 Accepted 2026-03-10 02:59:51,912 - lava_callback - INFO - Starting async job submit for node 69af67d2275c8d3b79399ef4 2026-03-10 02:59:51,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:59:51,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af67d2275c8d3b79399ef4 HTTP/1.1" 200 None 2026-03-10 02:59:51,951 - lava_callback - DEBUG - Processing node 69af67d2275c8d3b79399ef4: ltp-pty 2026-03-10 02:59:52,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:59:52,514 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:59:52,516 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-pty-69af67d2275c8d3b79399ef4/log.txt.gz 2026-03-10 02:59:52,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:59:52,939 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:59:52,941 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-pty-69af67d2275c8d3b79399ef4/lava-logs.yaml 2026-03-10 02:59:52,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:59:53,121 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:59:53,123 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-pty-69af67d2275c8d3b79399ef4/lava_callback.json.gz 2026-03-10 02:59:53,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:59:53,166 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af67d2275c8d3b79399ef4 HTTP/1.1" 200 None 2026-03-10 02:59:53,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:59:53,190 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af632c275c8d3b79399db4 HTTP/1.1" 200 None 2026-03-10 02:59:53,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:59:53,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af67d2275c8d3b79399ef4 HTTP/1.1" 200 None 2026-03-10 02:59:53,217 - lava_callback - INFO - Completed processing callback for node 69af67d2275c8d3b79399ef4 2026-03-10 02:59:53,437 - lava_callback - INFO - Received callback for node 69af67d4275c8d3b79399efb INFO: 172.20.0.1:57822 - "POST /node/69af67d4275c8d3b79399efb HTTP/1.0" 202 Accepted 2026-03-10 02:59:53,444 - lava_callback - INFO - Starting async job submit for node 69af67d4275c8d3b79399efb 2026-03-10 02:59:53,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:59:53,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:59:53,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-10 02:59:53,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af67d4275c8d3b79399efb HTTP/1.1" 200 None 2026-03-10 02:59:53,509 - lava_callback - DEBUG - Processing node 69af67d4275c8d3b79399efb: ltp-sched 2026-03-10 02:59:53,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:59:53,993 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:59:53,995 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-sched-69af67d4275c8d3b79399efb/log.txt.gz 2026-03-10 02:59:54,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:59:54,453 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:59:54,455 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-sched-69af67d4275c8d3b79399efb/lava-logs.yaml 2026-03-10 02:59:54,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 02:59:54,608 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 02:59:54,610 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-sched-69af67d4275c8d3b79399efb/lava_callback.json.gz 2026-03-10 02:59:54,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:59:54,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af67d4275c8d3b79399efb HTTP/1.1" 200 None 2026-03-10 02:59:54,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:59:54,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af632c275c8d3b79399db4 HTTP/1.1" 200 None 2026-03-10 02:59:54,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 02:59:54,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af67d4275c8d3b79399efb HTTP/1.1" 200 None 2026-03-10 02:59:54,693 - lava_callback - INFO - Completed processing callback for node 69af67d4275c8d3b79399efb INFO: 172.20.0.1:57838 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:00:04,725 - lava_callback - INFO - Received callback for node 69ace631bcd207e915d7810c INFO: 172.20.0.1:38494 - "POST /node/69ace631bcd207e915d7810c HTTP/1.0" 202 Accepted 2026-03-10 03:00:04,731 - lava_callback - INFO - Starting async job submit for node 69ace631bcd207e915d7810c 2026-03-10 03:00:04,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:00:04,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ace631bcd207e915d7810c HTTP/1.1" 200 None 2026-03-10 03:00:04,747 - lava_callback - DEBUG - Processing node 69ace631bcd207e915d7810c: baseline-arm-clabbe 2026-03-10 03:00:04,747 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ace631bcd207e915d7810c 2026-03-10 03:00:04,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:00:04,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ace631bcd207e915d7810c HTTP/1.1" 200 None 2026-03-10 03:00:04,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:00:04,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abdf2740deb775998f3ba4 HTTP/1.1" 200 None 2026-03-10 03:00:04,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:00:04,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ace631bcd207e915d7810c HTTP/1.1" 200 None 2026-03-10 03:00:04,809 - lava_callback - ERROR - Error processing callback for node 69ace631bcd207e915d7810c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47020 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47032 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:00:23,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:00:23,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:60250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41138 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:00:44,677 - lava_callback - INFO - Received callback for node 69ace659bcd207e915d78127 INFO: 172.20.0.1:46368 - "POST /node/69ace659bcd207e915d78127 HTTP/1.0" 202 Accepted 2026-03-10 03:00:44,682 - lava_callback - INFO - Starting async job submit for node 69ace659bcd207e915d78127 2026-03-10 03:00:44,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:00:44,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ace659bcd207e915d78127 HTTP/1.1" 200 None 2026-03-10 03:00:44,705 - lava_callback - DEBUG - Processing node 69ace659bcd207e915d78127: baseline-arm-clabbe 2026-03-10 03:00:44,705 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ace659bcd207e915d78127 2026-03-10 03:00:44,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:00:44,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ace659bcd207e915d78127 HTTP/1.1" 200 None 2026-03-10 03:00:44,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:00:44,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abdf2740deb775998f3ba4 HTTP/1.1" 200 None 2026-03-10 03:00:44,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:00:44,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ace659bcd207e915d78127 HTTP/1.1" 200 None 2026-03-10 03:00:44,764 - lava_callback - ERROR - Error processing callback for node 69ace659bcd207e915d78127: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-10 03:00:44,877 - lava_callback - INFO - Received callback for node 69af67d6275c8d3b79399efe INFO: 172.20.0.1:46372 - "POST /node/69af67d6275c8d3b79399efe HTTP/1.0" 202 Accepted 2026-03-10 03:00:44,885 - lava_callback - INFO - Starting async job submit for node 69af67d6275c8d3b79399efe 2026-03-10 03:00:44,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:00:44,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af67d6275c8d3b79399efe HTTP/1.1" 200 None 2026-03-10 03:00:44,933 - lava_callback - DEBUG - Processing node 69af67d6275c8d3b79399efe: ltp-smoke 2026-03-10 03:00:45,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:00:45,434 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:00:45,435 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69af67d6275c8d3b79399efe/log.txt.gz 2026-03-10 03:00:45,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:00:45,876 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:00:45,878 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69af67d6275c8d3b79399efe/lava-logs.yaml 2026-03-10 03:00:45,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:00:46,056 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:00:46,058 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69af67d6275c8d3b79399efe/lava_callback.json.gz 2026-03-10 03:00:46,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:00:46,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af67d6275c8d3b79399efe HTTP/1.1" 200 None 2026-03-10 03:00:46,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:00:46,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af632c275c8d3b79399db4 HTTP/1.1" 200 None 2026-03-10 03:00:46,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:00:46,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af67d6275c8d3b79399efe HTTP/1.1" 200 None 2026-03-10 03:00:46,150 - lava_callback - INFO - Completed processing callback for node 69af67d6275c8d3b79399efe INFO: 172.20.0.1:46374 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:00:53,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:00:53,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-10 03:00:56,400 - lava_callback - INFO - Received callback for node 69af8900275c8d3b7939ac70 INFO: 172.20.0.1:41460 - "POST /node/69af8900275c8d3b7939ac70 HTTP/1.0" 202 Accepted 2026-03-10 03:00:56,406 - lava_callback - INFO - Starting async job submit for node 69af8900275c8d3b7939ac70 2026-03-10 03:00:56,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:00:56,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af8900275c8d3b7939ac70 HTTP/1.1" 200 None 2026-03-10 03:00:56,457 - lava_callback - DEBUG - Processing node 69af8900275c8d3b7939ac70: baseline-nfs-arm64 INFO: 172.20.0.1:41474 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:00:56,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:00:57,031 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:00:57,033 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af8900275c8d3b7939ac70/log.txt.gz 2026-03-10 03:00:57,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:00:57,497 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:00:57,499 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af8900275c8d3b7939ac70/lava-logs.yaml 2026-03-10 03:00:57,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:00:57,670 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:00:57,672 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af8900275c8d3b7939ac70/lava_callback.json.gz DEBUG: Setup failed for 69af8900275c8d3b7939ac70. Transitting job node result: incomplete -> fail 2026-03-10 03:00:57,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:00:57,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af8900275c8d3b7939ac70 HTTP/1.1" 200 None 2026-03-10 03:00:57,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:00:57,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 03:00:57,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:00:57,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af8900275c8d3b7939ac70 HTTP/1.1" 200 None 2026-03-10 03:00:57,871 - lava_callback - INFO - Completed processing callback for node 69af8900275c8d3b7939ac70 INFO: 172.20.0.1:52846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48632 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48644 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:01:23,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:01:23,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:57964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52688 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53412 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:03:11,607 - lava_callback - INFO - Received callback for node 69af67d9275c8d3b79399f05 INFO: 172.20.0.1:52620 - "POST /node/69af67d9275c8d3b79399f05 HTTP/1.0" 202 Accepted 2026-03-10 03:03:11,613 - lava_callback - INFO - Starting async job submit for node 69af67d9275c8d3b79399f05 2026-03-10 03:03:11,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:03:11,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af67d9275c8d3b79399f05 HTTP/1.1" 200 None 2026-03-10 03:03:11,664 - lava_callback - DEBUG - Processing node 69af67d9275c8d3b79399f05: ltp-syscalls-ipc 2026-03-10 03:03:12,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:03:12,167 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:03:12,168 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69af67d9275c8d3b79399f05/log.txt.gz 2026-03-10 03:03:12,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:03:12,601 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:03:12,604 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69af67d9275c8d3b79399f05/lava-logs.yaml 2026-03-10 03:03:12,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:03:12,795 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:03:12,797 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69af67d9275c8d3b79399f05/lava_callback.json.gz 2026-03-10 03:03:12,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:03:12,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af67d9275c8d3b79399f05 HTTP/1.1" 200 None 2026-03-10 03:03:12,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:03:12,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af632c275c8d3b79399db4 HTTP/1.1" 200 None 2026-03-10 03:03:12,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:03:12,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af67d9275c8d3b79399f05 HTTP/1.1" 200 None 2026-03-10 03:03:12,932 - lava_callback - INFO - Completed processing callback for node 69af67d9275c8d3b79399f05 INFO: 172.20.0.1:52630 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52642 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:03:23,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:03:23,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:56036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55794 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:03:52,190 - lava_callback - INFO - Received callback for node 69af67dc275c8d3b79399f1f INFO: 172.20.0.1:42522 - "POST /node/69af67dc275c8d3b79399f1f HTTP/1.0" 202 Accepted 2026-03-10 03:03:52,196 - lava_callback - INFO - Starting async job submit for node 69af67dc275c8d3b79399f1f 2026-03-10 03:03:52,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:03:52,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af67dc275c8d3b79399f1f HTTP/1.1" 200 None 2026-03-10 03:03:52,241 - lava_callback - DEBUG - Processing node 69af67dc275c8d3b79399f1f: ltp-watchqueue 2026-03-10 03:03:52,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:03:52,696 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:03:52,698 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69af67dc275c8d3b79399f1f/log.txt.gz 2026-03-10 03:03:52,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:03:53,112 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:03:53,114 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69af67dc275c8d3b79399f1f/lava-logs.yaml 2026-03-10 03:03:53,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:03:53,287 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:03:53,289 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69af67dc275c8d3b79399f1f/lava_callback.json.gz 2026-03-10 03:03:53,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:03:53,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af67dc275c8d3b79399f1f HTTP/1.1" 200 None 2026-03-10 03:03:53,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:03:53,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af632c275c8d3b79399db4 HTTP/1.1" 200 None 2026-03-10 03:03:53,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:03:53,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af67dc275c8d3b79399f1f HTTP/1.1" 200 None 2026-03-10 03:03:53,375 - lava_callback - INFO - Completed processing callback for node 69af67dc275c8d3b79399f1f 2026-03-10 03:03:53,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:03:53,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:42524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35714 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32768 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:04:31,978 - lava_callback - INFO - Received callback for node 69af67da275c8d3b79399f1c INFO: 172.20.0.1:54930 - "POST /node/69af67da275c8d3b79399f1c HTTP/1.0" 202 Accepted 2026-03-10 03:04:31,985 - lava_callback - INFO - Starting async job submit for node 69af67da275c8d3b79399f1c 2026-03-10 03:04:32,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:04:32,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af67da275c8d3b79399f1c HTTP/1.1" 200 None 2026-03-10 03:04:32,025 - lava_callback - DEBUG - Processing node 69af67da275c8d3b79399f1c: ltp-timers 2026-03-10 03:04:32,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:04:32,585 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:04:32,587 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-timers-69af67da275c8d3b79399f1c/log.txt.gz 2026-03-10 03:04:32,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:04:33,051 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:04:33,053 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-timers-69af67da275c8d3b79399f1c/lava-logs.yaml 2026-03-10 03:04:33,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:04:33,245 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:04:33,247 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-timers-69af67da275c8d3b79399f1c/lava_callback.json.gz 2026-03-10 03:04:33,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:04:33,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af67da275c8d3b79399f1c HTTP/1.1" 200 None 2026-03-10 03:04:33,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:04:33,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af632c275c8d3b79399db4 HTTP/1.1" 200 None 2026-03-10 03:04:33,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:04:33,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af67da275c8d3b79399f1c HTTP/1.1" 200 None 2026-03-10 03:04:33,356 - lava_callback - INFO - Completed processing callback for node 69af67da275c8d3b79399f1c INFO: 172.20.0.1:54940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52232 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:04:53,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:04:53,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:43616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54638 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58022 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38078 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:07:14,130 - lava_callback - INFO - Received callback for node 69af8822275c8d3b7939ac2f INFO: 172.20.0.1:49314 - "POST /node/69af8822275c8d3b7939ac2f HTTP/1.0" 202 Accepted 2026-03-10 03:07:14,136 - lava_callback - INFO - Starting async job submit for node 69af8822275c8d3b7939ac2f 2026-03-10 03:07:14,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:07:14,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af8822275c8d3b7939ac2f HTTP/1.1" 200 None 2026-03-10 03:07:14,170 - lava_callback - DEBUG - Processing node 69af8822275c8d3b7939ac2f: baseline-arm64-foundriesio 2026-03-10 03:07:14,220 - lava_callback - WARNING - Failed to upload log 2026-03-10 03:07:14,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:07:14,432 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:07:14,434 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69af8822275c8d3b7939ac2f/lava-logs.yaml 2026-03-10 03:07:14,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:07:14,576 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:07:14,578 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69af8822275c8d3b7939ac2f/lava_callback.json.gz 2026-03-10 03:07:14,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:07:14,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af8822275c8d3b7939ac2f HTTP/1.1" 200 None 2026-03-10 03:07:14,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:07:14,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 03:07:14,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:07:14,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af8822275c8d3b7939ac2f HTTP/1.1" 200 None 2026-03-10 03:07:14,712 - lava_callback - INFO - Completed processing callback for node 69af8822275c8d3b7939ac2f INFO: 172.20.0.1:49326 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49332 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:07:23,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:07:23,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:44340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57020 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45806 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37552 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33966 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58928 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39058 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:12:39,834 - lava_callback - INFO - Received callback for node 69af8c11275c8d3b7939ad39 INFO: 172.20.0.1:33268 - "POST /node/69af8c11275c8d3b7939ad39 HTTP/1.0" 202 Accepted 2026-03-10 03:12:39,840 - lava_callback - INFO - Starting async job submit for node 69af8c11275c8d3b7939ad39 2026-03-10 03:12:39,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:12:39,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af8c11275c8d3b7939ad39 HTTP/1.1" 200 None 2026-03-10 03:12:39,872 - lava_callback - DEBUG - Processing node 69af8c11275c8d3b7939ad39: baseline-arm64 2026-03-10 03:12:39,920 - lava_callback - WARNING - Failed to upload log 2026-03-10 03:12:39,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:12:40,139 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:12:40,141 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af8c11275c8d3b7939ad39/lava-logs.yaml 2026-03-10 03:12:40,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:12:40,287 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:12:40,289 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af8c11275c8d3b7939ad39/lava_callback.json.gz 2026-03-10 03:12:40,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:12:40,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af8c11275c8d3b7939ad39 HTTP/1.1" 200 None 2026-03-10 03:12:40,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:12:40,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 03:12:40,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:12:40,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af8c11275c8d3b7939ad39 HTTP/1.1" 200 None 2026-03-10 03:12:40,390 - lava_callback - INFO - Completed processing callback for node 69af8c11275c8d3b7939ad39 INFO: 172.20.0.1:33272 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:12:53,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:12:53,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:58204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40494 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57516 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49608 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49614 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:15:18,436 - lava_callback - INFO - Received callback for node 69ace9c0bcd207e915d7820e INFO: 172.20.0.1:49616 - "POST /node/69ace9c0bcd207e915d7820e HTTP/1.0" 202 Accepted 2026-03-10 03:15:18,441 - lava_callback - INFO - Starting async job submit for node 69ace9c0bcd207e915d7820e 2026-03-10 03:15:18,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:15:18,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ace9c0bcd207e915d7820e HTTP/1.1" 200 None 2026-03-10 03:15:18,456 - lava_callback - DEBUG - Processing node 69ace9c0bcd207e915d7820e: baseline-arm64-baylibre 2026-03-10 03:15:18,457 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ace9c0bcd207e915d7820e 2026-03-10 03:15:18,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:15:18,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ace9c0bcd207e915d7820e HTTP/1.1" 200 None 2026-03-10 03:15:18,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:15:18,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a744905a0c0e10c4819cd5 HTTP/1.1" 200 None 2026-03-10 03:15:18,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:15:18,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ace9c0bcd207e915d7820e HTTP/1.1" 200 None 2026-03-10 03:15:18,512 - lava_callback - ERROR - Error processing callback for node 69ace9c0bcd207e915d7820e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49052 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38260 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:16:38,635 - lava_callback - INFO - Received callback for node 69acea11bcd207e915d7822e INFO: 172.20.0.1:38262 - "POST /node/69acea11bcd207e915d7822e HTTP/1.0" 202 Accepted 2026-03-10 03:16:38,640 - lava_callback - INFO - Starting async job submit for node 69acea11bcd207e915d7822e 2026-03-10 03:16:38,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:16:38,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acea11bcd207e915d7822e HTTP/1.1" 200 None 2026-03-10 03:16:38,656 - lava_callback - DEBUG - Processing node 69acea11bcd207e915d7822e: baseline-arm64-baylibre 2026-03-10 03:16:38,656 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acea11bcd207e915d7822e 2026-03-10 03:16:38,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:16:38,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acea11bcd207e915d7822e HTTP/1.1" 200 None 2026-03-10 03:16:38,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:16:38,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a744905a0c0e10c4819cd5 HTTP/1.1" 200 None 2026-03-10 03:16:38,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:16:38,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acea11bcd207e915d7822e HTTP/1.1" 200 None 2026-03-10 03:16:38,707 - lava_callback - ERROR - Error processing callback for node 69acea11bcd207e915d7822e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58070 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60942 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36048 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:19:12,530 - lava_callback - INFO - Received callback for node 69af8c72275c8d3b7939ad4a INFO: 172.20.0.1:44690 - "POST /node/69af8c72275c8d3b7939ad4a HTTP/1.0" 202 Accepted 2026-03-10 03:19:12,537 - lava_callback - INFO - Starting async job submit for node 69af8c72275c8d3b7939ad4a 2026-03-10 03:19:12,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:19:12,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af8c72275c8d3b7939ad4a HTTP/1.1" 200 None 2026-03-10 03:19:12,590 - lava_callback - DEBUG - Processing node 69af8c72275c8d3b7939ad4a: baseline-arm64-broonie 2026-03-10 03:19:13,030 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:19:13,215 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:19:13,217 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af8c72275c8d3b7939ad4a/log.txt.gz 2026-03-10 03:19:13,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:19:13,705 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:19:13,707 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af8c72275c8d3b7939ad4a/lava-logs.yaml 2026-03-10 03:19:13,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:19:13,893 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:19:13,895 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af8c72275c8d3b7939ad4a/lava_callback.json.gz DEBUG: Setup failed for 69af8c72275c8d3b7939ad4a. Transitting job node result: incomplete -> fail 2026-03-10 03:19:13,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:19:13,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af8c72275c8d3b7939ad4a HTTP/1.1" 200 None 2026-03-10 03:19:13,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:19:14,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 03:19:14,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:19:14,052 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af8c72275c8d3b7939ad4a HTTP/1.1" 200 None 2026-03-10 03:19:14,144 - lava_callback - INFO - Completed processing callback for node 69af8c72275c8d3b7939ad4a INFO: 172.20.0.1:44702 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44716 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:19:18,632 - lava_callback - INFO - Received callback for node 69aceaabbcd207e915d78258 INFO: 172.20.0.1:44718 - "POST /node/69aceaabbcd207e915d78258 HTTP/1.0" 202 Accepted 2026-03-10 03:19:18,638 - lava_callback - INFO - Starting async job submit for node 69aceaabbcd207e915d78258 2026-03-10 03:19:18,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:19:18,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aceaabbcd207e915d78258 HTTP/1.1" 200 None 2026-03-10 03:19:18,656 - lava_callback - DEBUG - Processing node 69aceaabbcd207e915d78258: baseline-arm64-baylibre 2026-03-10 03:19:18,656 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69aceaabbcd207e915d78258 2026-03-10 03:19:18,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:19:18,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aceaabbcd207e915d78258 HTTP/1.1" 200 None 2026-03-10 03:19:18,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:19:18,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a744905a0c0e10c4819cd5 HTTP/1.1" 200 None 2026-03-10 03:19:18,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:19:18,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aceaabbcd207e915d78258 HTTP/1.1" 200 None 2026-03-10 03:19:18,724 - lava_callback - ERROR - Error processing callback for node 69aceaabbcd207e915d78258: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-10 03:19:23,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:19:23,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:38380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51458 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:19:44,508 - lava_callback - INFO - Received callback for node 69af8d7c275c8d3b7939ad84 INFO: 172.20.0.1:40750 - "POST /node/69af8d7c275c8d3b7939ad84 HTTP/1.0" 202 Accepted 2026-03-10 03:19:44,515 - lava_callback - INFO - Starting async job submit for node 69af8d7c275c8d3b7939ad84 2026-03-10 03:19:44,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:19:44,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af8d7c275c8d3b7939ad84 HTTP/1.1" 200 None 2026-03-10 03:19:44,561 - lava_callback - DEBUG - Processing node 69af8d7c275c8d3b7939ad84: baseline-arm64 2026-03-10 03:19:45,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:19:45,512 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:19:45,514 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af8d7c275c8d3b7939ad84/log.txt.gz 2026-03-10 03:19:45,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:19:46,164 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:19:46,167 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af8d7c275c8d3b7939ad84/lava-logs.yaml 2026-03-10 03:19:46,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:19:46,343 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:19:46,345 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af8d7c275c8d3b7939ad84/lava_callback.json.gz DEBUG: Setup failed for 69af8d7c275c8d3b7939ad84. Transitting job node result: incomplete -> fail 2026-03-10 03:19:46,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:19:46,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af8d7c275c8d3b7939ad84 HTTP/1.1" 200 None 2026-03-10 03:19:46,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:19:46,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 03:19:46,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:19:46,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af8d7c275c8d3b7939ad84 HTTP/1.1" 200 None INFO: 172.20.0.1:40766 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:19:46,558 - lava_callback - INFO - Completed processing callback for node 69af8d7c275c8d3b7939ad84 2026-03-10 03:19:53,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:19:53,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:38104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40576 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44382 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:20:38,738 - lava_callback - INFO - Received callback for node 69aceaf4bcd207e915d78276 INFO: 172.20.0.1:44386 - "POST /node/69aceaf4bcd207e915d78276 HTTP/1.0" 202 Accepted 2026-03-10 03:20:38,744 - lava_callback - INFO - Starting async job submit for node 69aceaf4bcd207e915d78276 2026-03-10 03:20:38,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:20:38,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aceaf4bcd207e915d78276 HTTP/1.1" 200 None 2026-03-10 03:20:38,767 - lava_callback - DEBUG - Processing node 69aceaf4bcd207e915d78276: baseline-arm64-baylibre 2026-03-10 03:20:38,767 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69aceaf4bcd207e915d78276 2026-03-10 03:20:38,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:20:38,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aceaf4bcd207e915d78276 HTTP/1.1" 200 None 2026-03-10 03:20:38,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:20:38,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abdf3340deb775998f3bb5 HTTP/1.1" 200 None 2026-03-10 03:20:38,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:20:38,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aceaf4bcd207e915d78276 HTTP/1.1" 200 None 2026-03-10 03:20:38,875 - lava_callback - ERROR - Error processing callback for node 69aceaf4bcd207e915d78276: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36368 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59092 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:21:58,835 - lava_callback - INFO - Received callback for node 69aceb44bcd207e915d78289 INFO: 172.20.0.1:59108 - "POST /node/69aceb44bcd207e915d78289 HTTP/1.0" 202 Accepted 2026-03-10 03:21:58,842 - lava_callback - INFO - Starting async job submit for node 69aceb44bcd207e915d78289 2026-03-10 03:21:58,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:21:58,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aceb44bcd207e915d78289 HTTP/1.1" 200 None 2026-03-10 03:21:58,863 - lava_callback - DEBUG - Processing node 69aceb44bcd207e915d78289: baseline-arm64-baylibre 2026-03-10 03:21:58,864 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69aceb44bcd207e915d78289 2026-03-10 03:21:58,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:21:58,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aceb44bcd207e915d78289 HTTP/1.1" 200 None 2026-03-10 03:21:58,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:21:58,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a744905a0c0e10c4819cd5 HTTP/1.1" 200 None 2026-03-10 03:21:58,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:21:58,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aceb44bcd207e915d78289 HTTP/1.1" 200 None 2026-03-10 03:21:58,922 - lava_callback - ERROR - Error processing callback for node 69aceb44bcd207e915d78289: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34042 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40220 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40234 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:23:18,185 - lava_callback - INFO - Received callback for node 69af8e43275c8d3b7939adb4 INFO: 172.20.0.1:40248 - "POST /node/69af8e43275c8d3b7939adb4 HTTP/1.0" 202 Accepted 2026-03-10 03:23:18,193 - lava_callback - INFO - Starting async job submit for node 69af8e43275c8d3b7939adb4 2026-03-10 03:23:18,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:23:18,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af8e43275c8d3b7939adb4 HTTP/1.1" 200 None 2026-03-10 03:23:18,241 - lava_callback - DEBUG - Processing node 69af8e43275c8d3b7939adb4: baseline-nfs-arm64 2026-03-10 03:23:18,920 - lava_callback - INFO - Received callback for node 69aceb94bcd207e915d7829f INFO: 172.20.0.1:40256 - "POST /node/69aceb94bcd207e915d7829f HTTP/1.0" 202 Accepted 2026-03-10 03:23:18,936 - lava_callback - INFO - Starting async job submit for node 69aceb94bcd207e915d7829f 2026-03-10 03:23:18,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:23:19,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aceb94bcd207e915d7829f HTTP/1.1" 200 None 2026-03-10 03:23:19,042 - lava_callback - DEBUG - Processing node 69aceb94bcd207e915d7829f: baseline-arm64-baylibre 2026-03-10 03:23:19,043 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69aceb94bcd207e915d7829f 2026-03-10 03:23:19,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:23:19,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:23:19,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aceb94bcd207e915d7829f HTTP/1.1" 200 None 2026-03-10 03:23:19,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:23:19,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abdf3340deb775998f3bb5 HTTP/1.1" 200 None 2026-03-10 03:23:19,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:23:19,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aceb94bcd207e915d7829f HTTP/1.1" 200 None 2026-03-10 03:23:19,142 - lava_callback - ERROR - Error processing callback for node 69aceb94bcd207e915d7829f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-10 03:23:19,306 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:23:19,308 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af8e43275c8d3b7939adb4/log.txt.gz 2026-03-10 03:23:19,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:23:20,071 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:23:20,073 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af8e43275c8d3b7939adb4/lava-logs.yaml 2026-03-10 03:23:20,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:23:20,266 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:23:20,268 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af8e43275c8d3b7939adb4/lava_callback.json.gz DEBUG: Setup failed for 69af8e43275c8d3b7939adb4. Transitting job node result: incomplete -> fail 2026-03-10 03:23:20,334 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:23:20,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af8e43275c8d3b7939adb4 HTTP/1.1" 200 None 2026-03-10 03:23:20,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:23:20,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 03:23:20,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:23:20,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af8e43275c8d3b7939adb4 HTTP/1.1" 200 None 2026-03-10 03:23:20,467 - lava_callback - INFO - Completed processing callback for node 69af8e43275c8d3b7939adb4 2026-03-10 03:23:23,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:23:23,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:33482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49116 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56124 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:24:38,950 - lava_callback - INFO - Received callback for node 69acebe6bcd207e915d782bf INFO: 172.20.0.1:56140 - "POST /node/69acebe6bcd207e915d782bf HTTP/1.0" 202 Accepted 2026-03-10 03:24:38,955 - lava_callback - INFO - Starting async job submit for node 69acebe6bcd207e915d782bf 2026-03-10 03:24:38,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:24:38,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acebe6bcd207e915d782bf HTTP/1.1" 200 None 2026-03-10 03:24:38,971 - lava_callback - DEBUG - Processing node 69acebe6bcd207e915d782bf: baseline-arm64-baylibre 2026-03-10 03:24:38,972 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acebe6bcd207e915d782bf 2026-03-10 03:24:38,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:24:38,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acebe6bcd207e915d782bf HTTP/1.1" 200 None 2026-03-10 03:24:38,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:24:39,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a744905a0c0e10c4819cd5 HTTP/1.1" 200 None 2026-03-10 03:24:39,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:24:39,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acebe6bcd207e915d782bf HTTP/1.1" 200 None 2026-03-10 03:24:39,036 - lava_callback - ERROR - Error processing callback for node 69acebe6bcd207e915d782bf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-10 03:24:44,072 - lava_callback - INFO - Received callback for node 69af8edb275c8d3b7939adda INFO: 172.20.0.1:44176 - "POST /node/69af8edb275c8d3b7939adda HTTP/1.0" 202 Accepted 2026-03-10 03:24:44,079 - lava_callback - INFO - Starting async job submit for node 69af8edb275c8d3b7939adda 2026-03-10 03:24:44,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:24:44,110 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af8edb275c8d3b7939adda HTTP/1.1" 200 None 2026-03-10 03:24:44,111 - lava_callback - DEBUG - Processing node 69af8edb275c8d3b7939adda: baseline-arm64 2026-03-10 03:24:44,158 - lava_callback - WARNING - Failed to upload log 2026-03-10 03:24:44,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:24:44,338 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:24:44,340 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af8edb275c8d3b7939adda/lava-logs.yaml 2026-03-10 03:24:44,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:24:44,487 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:24:44,489 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af8edb275c8d3b7939adda/lava_callback.json.gz 2026-03-10 03:24:44,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:24:44,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af8edb275c8d3b7939adda HTTP/1.1" 200 None 2026-03-10 03:24:44,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:24:44,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 03:24:44,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:24:44,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af8edb275c8d3b7939adda HTTP/1.1" 200 None 2026-03-10 03:24:44,627 - lava_callback - INFO - Completed processing callback for node 69af8edb275c8d3b7939adda INFO: 172.20.0.1:44178 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:24:53,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:24:53,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:33372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54098 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48638 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48650 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:26:19,033 - lava_callback - INFO - Received callback for node 69acec56bcd207e915d782db INFO: 172.20.0.1:48656 - "POST /node/69acec56bcd207e915d782db HTTP/1.0" 202 Accepted 2026-03-10 03:26:19,040 - lava_callback - INFO - Starting async job submit for node 69acec56bcd207e915d782db 2026-03-10 03:26:19,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:26:19,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acec56bcd207e915d782db HTTP/1.1" 200 None 2026-03-10 03:26:19,058 - lava_callback - DEBUG - Processing node 69acec56bcd207e915d782db: baseline-arm64-baylibre 2026-03-10 03:26:19,058 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acec56bcd207e915d782db 2026-03-10 03:26:19,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:26:19,082 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acec56bcd207e915d782db HTTP/1.1" 200 None 2026-03-10 03:26:19,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:26:19,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a744905a0c0e10c4819cd5 HTTP/1.1" 200 None 2026-03-10 03:26:19,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:26:19,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acec56bcd207e915d782db HTTP/1.1" 200 None 2026-03-10 03:26:19,125 - lava_callback - ERROR - Error processing callback for node 69acec56bcd207e915d782db: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57292 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38440 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35614 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35618 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:29:19,217 - lava_callback - INFO - Received callback for node 69aced0abcd207e915d7830a INFO: 172.20.0.1:35622 - "POST /node/69aced0abcd207e915d7830a HTTP/1.0" 202 Accepted 2026-03-10 03:29:19,222 - lava_callback - INFO - Starting async job submit for node 69aced0abcd207e915d7830a 2026-03-10 03:29:19,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:29:19,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aced0abcd207e915d7830a HTTP/1.1" 200 None 2026-03-10 03:29:19,238 - lava_callback - DEBUG - Processing node 69aced0abcd207e915d7830a: baseline-arm64-baylibre 2026-03-10 03:29:19,239 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69aced0abcd207e915d7830a 2026-03-10 03:29:19,239 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:29:19,258 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aced0abcd207e915d7830a HTTP/1.1" 200 None 2026-03-10 03:29:19,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:29:19,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a744905a0c0e10c4819cd5 HTTP/1.1" 200 None 2026-03-10 03:29:19,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:29:19,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aced0abcd207e915d7830a HTTP/1.1" 200 None 2026-03-10 03:29:19,297 - lava_callback - ERROR - Error processing callback for node 69aced0abcd207e915d7830a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36104 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:29:38,144 - lava_callback - INFO - Received callback for node 69af9000275c8d3b7939ae1c INFO: 172.20.0.1:36120 - "POST /node/69af9000275c8d3b7939ae1c HTTP/1.0" 202 Accepted 2026-03-10 03:29:38,150 - lava_callback - INFO - Starting async job submit for node 69af9000275c8d3b7939ae1c 2026-03-10 03:29:38,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:29:38,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af9000275c8d3b7939ae1c HTTP/1.1" 200 None 2026-03-10 03:29:38,166 - lava_callback - DEBUG - Processing node 69af9000275c8d3b7939ae1c: baseline-nfs-arm64 2026-03-10 03:29:38,168 - lava_callback - WARNING - Failed to upload log 2026-03-10 03:29:38,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:29:38,341 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:29:38,343 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af9000275c8d3b7939ae1c/lava-logs.yaml 2026-03-10 03:29:38,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:29:38,473 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:29:38,475 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af9000275c8d3b7939ae1c/lava_callback.json.gz 2026-03-10 03:29:38,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:29:38,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af9000275c8d3b7939ae1c HTTP/1.1" 200 None 2026-03-10 03:29:38,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:29:38,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 03:29:38,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:29:38,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af9000275c8d3b7939ae1c HTTP/1.1" 200 None 2026-03-10 03:29:38,535 - lava_callback - INFO - Completed processing callback for node 69af9000275c8d3b7939ae1c INFO: 172.20.0.1:39480 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:29:53,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:29:53,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:40410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35176 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60048 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:30:34,405 - lava_callback - INFO - Received callback for node 69af8a4d275c8d3b7939ace7 INFO: 172.20.0.1:47234 - "POST /node/69af8a4d275c8d3b7939ace7 HTTP/1.0" 202 Accepted 2026-03-10 03:30:34,411 - lava_callback - INFO - Starting async job submit for node 69af8a4d275c8d3b7939ace7 2026-03-10 03:30:34,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:30:34,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af8a4d275c8d3b7939ace7 HTTP/1.1" 200 None 2026-03-10 03:30:34,462 - lava_callback - DEBUG - Processing node 69af8a4d275c8d3b7939ace7: baseline-arm64-broonie 2026-03-10 03:30:34,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:30:35,012 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:30:35,013 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af8a4d275c8d3b7939ace7/log.txt.gz 2026-03-10 03:30:35,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:30:35,455 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:30:35,457 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af8a4d275c8d3b7939ace7/lava-logs.yaml 2026-03-10 03:30:35,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:30:35,607 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:30:35,609 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af8a4d275c8d3b7939ace7/lava_callback.json.gz DEBUG: Setup failed for 69af8a4d275c8d3b7939ace7. Transitting job node result: incomplete -> fail 2026-03-10 03:30:35,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:30:35,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af8a4d275c8d3b7939ace7 HTTP/1.1" 200 None 2026-03-10 03:30:35,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:30:35,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 03:30:35,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:30:35,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af8a4d275c8d3b7939ace7 HTTP/1.1" 200 None 2026-03-10 03:30:35,815 - lava_callback - INFO - Completed processing callback for node 69af8a4d275c8d3b7939ace7 INFO: 172.20.0.1:47250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32794 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:30:53,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:30:53,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:38272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40700 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40706 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:31:19,266 - lava_callback - INFO - Received callback for node 69aced83bcd207e915d78327 INFO: 172.20.0.1:40716 - "POST /node/69aced83bcd207e915d78327 HTTP/1.0" 202 Accepted 2026-03-10 03:31:19,271 - lava_callback - INFO - Starting async job submit for node 69aced83bcd207e915d78327 2026-03-10 03:31:19,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:31:19,286 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aced83bcd207e915d78327 HTTP/1.1" 200 None 2026-03-10 03:31:19,287 - lava_callback - DEBUG - Processing node 69aced83bcd207e915d78327: baseline-arm64-baylibre 2026-03-10 03:31:19,287 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69aced83bcd207e915d78327 2026-03-10 03:31:19,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:31:19,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aced83bcd207e915d78327 HTTP/1.1" 200 None 2026-03-10 03:31:19,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:31:19,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abdf3340deb775998f3bb5 HTTP/1.1" 200 None 2026-03-10 03:31:19,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:31:19,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aced83bcd207e915d78327 HTTP/1.1" 200 None 2026-03-10 03:31:19,339 - lava_callback - ERROR - Error processing callback for node 69aced83bcd207e915d78327: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41462 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:31:27,471 - lava_callback - INFO - Received callback for node 69af9074275c8d3b7939ae38 INFO: 172.20.0.1:41464 - "POST /node/69af9074275c8d3b7939ae38 HTTP/1.0" 202 Accepted 2026-03-10 03:31:27,478 - lava_callback - INFO - Starting async job submit for node 69af9074275c8d3b7939ae38 2026-03-10 03:31:27,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:31:27,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af9074275c8d3b7939ae38 HTTP/1.1" 200 None 2026-03-10 03:31:27,510 - lava_callback - DEBUG - Processing node 69af9074275c8d3b7939ae38: baseline-arm64 2026-03-10 03:31:27,560 - lava_callback - WARNING - Failed to upload log 2026-03-10 03:31:27,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:31:27,751 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:31:27,753 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af9074275c8d3b7939ae38/lava-logs.yaml 2026-03-10 03:31:27,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:31:27,893 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:31:27,894 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af9074275c8d3b7939ae38/lava_callback.json.gz 2026-03-10 03:31:27,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:31:27,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af9074275c8d3b7939ae38 HTTP/1.1" 200 None 2026-03-10 03:31:27,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:31:27,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 03:31:27,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:31:27,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af9074275c8d3b7939ae38 HTTP/1.1" 200 None 2026-03-10 03:31:28,019 - lava_callback - INFO - Completed processing callback for node 69af9074275c8d3b7939ae38 INFO: 172.20.0.1:56356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44376 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:31:53,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:31:53,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:35588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59454 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58132 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:32:46,085 - lava_callback - INFO - Received callback for node 69aceddbbcd207e915d78341 INFO: 172.20.0.1:34828 - "POST /node/69aceddbbcd207e915d78341 HTTP/1.0" 202 Accepted 2026-03-10 03:32:46,091 - lava_callback - INFO - Starting async job submit for node 69aceddbbcd207e915d78341 2026-03-10 03:32:46,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:32:46,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aceddbbcd207e915d78341 HTTP/1.1" 200 None 2026-03-10 03:32:46,112 - lava_callback - DEBUG - Processing node 69aceddbbcd207e915d78341: baseline-arm64-clabbe 2026-03-10 03:32:46,112 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69aceddbbcd207e915d78341 2026-03-10 03:32:46,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:32:46,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aceddbbcd207e915d78341 HTTP/1.1" 200 None 2026-03-10 03:32:46,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:32:46,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abdf3340deb775998f3bb5 HTTP/1.1" 200 None 2026-03-10 03:32:46,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:32:46,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aceddbbcd207e915d78341 HTTP/1.1" 200 None 2026-03-10 03:32:46,163 - lava_callback - ERROR - Error processing callback for node 69aceddbbcd207e915d78341: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52472 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37222 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37238 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:34:26,135 - lava_callback - INFO - Received callback for node 69acee3ebcd207e915d78366 INFO: 172.20.0.1:58566 - "POST /node/69acee3ebcd207e915d78366 HTTP/1.0" 202 Accepted 2026-03-10 03:34:26,141 - lava_callback - INFO - Starting async job submit for node 69acee3ebcd207e915d78366 2026-03-10 03:34:26,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:34:26,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acee3ebcd207e915d78366 HTTP/1.1" 200 None 2026-03-10 03:34:26,157 - lava_callback - DEBUG - Processing node 69acee3ebcd207e915d78366: baseline-arm64-clabbe 2026-03-10 03:34:26,157 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acee3ebcd207e915d78366 2026-03-10 03:34:26,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:34:26,190 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acee3ebcd207e915d78366 HTTP/1.1" 200 None 2026-03-10 03:34:26,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:34:26,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abdf3340deb775998f3bb5 HTTP/1.1" 200 None 2026-03-10 03:34:26,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:34:26,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acee3ebcd207e915d78366 HTTP/1.1" 200 None 2026-03-10 03:34:26,232 - lava_callback - ERROR - Error processing callback for node 69acee3ebcd207e915d78366: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60906 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56134 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:35:40,005 - lava_callback - INFO - Received callback for node 69acee7bbcd207e915d78389 INFO: 172.20.0.1:55076 - "POST /node/69acee7bbcd207e915d78389 HTTP/1.0" 202 Accepted 2026-03-10 03:35:40,012 - lava_callback - INFO - Starting async job submit for node 69acee7bbcd207e915d78389 2026-03-10 03:35:40,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:35:40,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acee7bbcd207e915d78389 HTTP/1.1" 200 None 2026-03-10 03:35:40,043 - lava_callback - DEBUG - Processing node 69acee7bbcd207e915d78389: baseline-arm-baylibre 2026-03-10 03:35:40,043 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acee7bbcd207e915d78389 2026-03-10 03:35:40,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:35:40,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acee7bbcd207e915d78389 HTTP/1.1" 200 None 2026-03-10 03:35:40,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:35:40,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a7447b5a0c0e10c4819cc3 HTTP/1.1" 200 None 2026-03-10 03:35:40,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:35:40,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acee7bbcd207e915d78389 HTTP/1.1" 200 None 2026-03-10 03:35:40,096 - lava_callback - ERROR - Error processing callback for node 69acee7bbcd207e915d78389: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47668 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47684 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:36:19,517 - lava_callback - INFO - Received callback for node 69aceea1bcd207e915d7839e INFO: 172.20.0.1:47698 - "POST /node/69aceea1bcd207e915d7839e HTTP/1.0" 202 Accepted 2026-03-10 03:36:19,522 - lava_callback - INFO - Starting async job submit for node 69aceea1bcd207e915d7839e 2026-03-10 03:36:19,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:36:19,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aceea1bcd207e915d7839e HTTP/1.1" 200 None 2026-03-10 03:36:19,539 - lava_callback - DEBUG - Processing node 69aceea1bcd207e915d7839e: baseline-arm-baylibre 2026-03-10 03:36:19,539 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69aceea1bcd207e915d7839e 2026-03-10 03:36:19,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:36:19,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aceea1bcd207e915d7839e HTTP/1.1" 200 None 2026-03-10 03:36:19,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:36:19,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a7447b5a0c0e10c4819cc3 HTTP/1.1" 200 None 2026-03-10 03:36:19,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:36:19,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aceea1bcd207e915d7839e HTTP/1.1" 200 None 2026-03-10 03:36:19,596 - lava_callback - ERROR - Error processing callback for node 69aceea1bcd207e915d7839e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51530 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:36:52,757 - lava_callback - INFO - Received callback for node 69af91b8275c8d3b7939ae71 INFO: 172.20.0.1:48096 - "POST /node/69af91b8275c8d3b7939ae71 HTTP/1.0" 202 Accepted 2026-03-10 03:36:52,762 - lava_callback - INFO - Starting async job submit for node 69af91b8275c8d3b7939ae71 2026-03-10 03:36:52,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:36:52,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af91b8275c8d3b7939ae71 HTTP/1.1" 200 None 2026-03-10 03:36:52,780 - lava_callback - DEBUG - Processing node 69af91b8275c8d3b7939ae71: baseline-arm64-broonie 2026-03-10 03:36:52,785 - lava_callback - WARNING - Failed to upload log 2026-03-10 03:36:52,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:36:52,958 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:36:52,959 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af91b8275c8d3b7939ae71/lava-logs.yaml 2026-03-10 03:36:52,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:36:53,108 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:36:53,110 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af91b8275c8d3b7939ae71/lava_callback.json.gz 2026-03-10 03:36:53,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:36:53,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af91b8275c8d3b7939ae71 HTTP/1.1" 200 None 2026-03-10 03:36:53,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:36:53,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 03:36:53,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:36:53,195 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af91b8275c8d3b7939ae71 HTTP/1.1" 200 None 2026-03-10 03:36:53,214 - lava_callback - INFO - Completed processing callback for node 69af91b8275c8d3b7939ae71 2026-03-10 03:36:53,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:36:53,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:48100 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:36:59,618 - lava_callback - INFO - Received callback for node 69aceec8bcd207e915d783af INFO: 172.20.0.1:48112 - "POST /node/69aceec8bcd207e915d783af HTTP/1.0" 202 Accepted 2026-03-10 03:36:59,624 - lava_callback - INFO - Starting async job submit for node 69aceec8bcd207e915d783af 2026-03-10 03:36:59,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:36:59,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aceec8bcd207e915d783af HTTP/1.1" 200 None 2026-03-10 03:36:59,642 - lava_callback - DEBUG - Processing node 69aceec8bcd207e915d783af: baseline-arm-baylibre 2026-03-10 03:36:59,642 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69aceec8bcd207e915d783af 2026-03-10 03:36:59,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:36:59,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aceec8bcd207e915d783af HTTP/1.1" 200 None 2026-03-10 03:36:59,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:36:59,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a7447b5a0c0e10c4819cc3 HTTP/1.1" 200 None 2026-03-10 03:36:59,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:36:59,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aceec8bcd207e915d783af HTTP/1.1" 200 None 2026-03-10 03:36:59,696 - lava_callback - ERROR - Error processing callback for node 69aceec8bcd207e915d783af: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42368 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51390 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:37:39,645 - lava_callback - INFO - Received callback for node 69aceef0bcd207e915d783c8 INFO: 172.20.0.1:51396 - "POST /node/69aceef0bcd207e915d783c8 HTTP/1.0" 202 Accepted 2026-03-10 03:37:39,662 - lava_callback - INFO - Starting async job submit for node 69aceef0bcd207e915d783c8 2026-03-10 03:37:39,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:37:39,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aceef0bcd207e915d783c8 HTTP/1.1" 200 None 2026-03-10 03:37:39,698 - lava_callback - DEBUG - Processing node 69aceef0bcd207e915d783c8: baseline-arm-baylibre 2026-03-10 03:37:39,698 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69aceef0bcd207e915d783c8 2026-03-10 03:37:39,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:37:39,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aceef0bcd207e915d783c8 HTTP/1.1" 200 None 2026-03-10 03:37:39,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:37:39,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a7447b5a0c0e10c4819cc3 HTTP/1.1" 200 None 2026-03-10 03:37:39,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:37:39,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aceef0bcd207e915d783c8 HTTP/1.1" 200 None 2026-03-10 03:37:39,762 - lava_callback - ERROR - Error processing callback for node 69aceef0bcd207e915d783c8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55918 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55926 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:38:19,688 - lava_callback - INFO - Received callback for node 69acef1cbcd207e915d783df INFO: 172.20.0.1:55928 - "POST /node/69acef1cbcd207e915d783df HTTP/1.0" 202 Accepted 2026-03-10 03:38:19,693 - lava_callback - INFO - Starting async job submit for node 69acef1cbcd207e915d783df 2026-03-10 03:38:19,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:38:19,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acef1cbcd207e915d783df HTTP/1.1" 200 None 2026-03-10 03:38:19,709 - lava_callback - DEBUG - Processing node 69acef1cbcd207e915d783df: baseline-arm-baylibre 2026-03-10 03:38:19,710 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acef1cbcd207e915d783df 2026-03-10 03:38:19,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:38:19,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acef1cbcd207e915d783df HTTP/1.1" 200 None 2026-03-10 03:38:19,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:38:19,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a7447b5a0c0e10c4819cc3 HTTP/1.1" 200 None 2026-03-10 03:38:19,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:38:19,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acef1cbcd207e915d783df HTTP/1.1" 200 None 2026-03-10 03:38:19,794 - lava_callback - ERROR - Error processing callback for node 69acef1cbcd207e915d783df: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39782 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:38:59,682 - lava_callback - INFO - Received callback for node 69acef43bcd207e915d783f7 INFO: 172.20.0.1:39794 - "POST /node/69acef43bcd207e915d783f7 HTTP/1.0" 202 Accepted 2026-03-10 03:38:59,688 - lava_callback - INFO - Starting async job submit for node 69acef43bcd207e915d783f7 2026-03-10 03:38:59,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:38:59,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acef43bcd207e915d783f7 HTTP/1.1" 200 None 2026-03-10 03:38:59,704 - lava_callback - DEBUG - Processing node 69acef43bcd207e915d783f7: baseline-arm-baylibre 2026-03-10 03:38:59,705 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acef43bcd207e915d783f7 2026-03-10 03:38:59,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:38:59,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acef43bcd207e915d783f7 HTTP/1.1" 200 None 2026-03-10 03:38:59,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:38:59,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a7447b5a0c0e10c4819cc3 HTTP/1.1" 200 None 2026-03-10 03:38:59,743 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:38:59,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acef43bcd207e915d783f7 HTTP/1.1" 200 None 2026-03-10 03:38:59,766 - lava_callback - ERROR - Error processing callback for node 69acef43bcd207e915d783f7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54352 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:39:06,424 - lava_callback - INFO - Received callback for node 69af895b275c8d3b7939ac95 INFO: 172.20.0.1:54366 - "POST /node/69af895b275c8d3b7939ac95 HTTP/1.0" 202 Accepted 2026-03-10 03:39:06,430 - lava_callback - INFO - Starting async job submit for node 69af895b275c8d3b7939ac95 2026-03-10 03:39:06,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:39:06,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af895b275c8d3b7939ac95 HTTP/1.1" 200 None 2026-03-10 03:39:06,481 - lava_callback - DEBUG - Processing node 69af895b275c8d3b7939ac95: baseline-arm64-broonie INFO: 172.20.0.1:54380 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:39:06,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:39:06,875 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:39:06,877 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af895b275c8d3b7939ac95/log.txt.gz 2026-03-10 03:39:07,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:39:07,220 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:39:07,222 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af895b275c8d3b7939ac95/lava-logs.yaml 2026-03-10 03:39:07,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:39:07,374 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:39:07,375 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af895b275c8d3b7939ac95/lava_callback.json.gz DEBUG: Setup failed for 69af895b275c8d3b7939ac95. Transitting job node result: incomplete -> fail 2026-03-10 03:39:07,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:39:07,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af895b275c8d3b7939ac95 HTTP/1.1" 200 None 2026-03-10 03:39:07,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:39:07,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 03:39:07,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:39:07,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af895b275c8d3b7939ac95 HTTP/1.1" 200 None 2026-03-10 03:39:07,605 - lava_callback - INFO - Completed processing callback for node 69af895b275c8d3b7939ac95 2026-03-10 03:39:12,812 - lava_callback - INFO - Received callback for node 69af9257275c8d3b7939ae8f INFO: 172.20.0.1:34052 - "POST /node/69af9257275c8d3b7939ae8f HTTP/1.0" 202 Accepted 2026-03-10 03:39:12,818 - lava_callback - INFO - Starting async job submit for node 69af9257275c8d3b7939ae8f 2026-03-10 03:39:12,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:39:12,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af9257275c8d3b7939ae8f HTTP/1.1" 200 None 2026-03-10 03:39:12,836 - lava_callback - DEBUG - Processing node 69af9257275c8d3b7939ae8f: baseline-arm64-clabbe 2026-03-10 03:39:12,840 - lava_callback - WARNING - Failed to upload log 2026-03-10 03:39:12,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:39:12,975 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:39:12,977 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69af9257275c8d3b7939ae8f/lava-logs.yaml 2026-03-10 03:39:12,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:39:13,130 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:39:13,131 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69af9257275c8d3b7939ae8f/lava_callback.json.gz 2026-03-10 03:39:13,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:39:13,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af9257275c8d3b7939ae8f HTTP/1.1" 200 None 2026-03-10 03:39:13,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:39:13,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 03:39:13,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:39:13,185 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af9257275c8d3b7939ae8f HTTP/1.1" 200 None 2026-03-10 03:39:13,195 - lava_callback - INFO - Completed processing callback for node 69af9257275c8d3b7939ae8f INFO: 172.20.0.1:34058 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34064 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:39:23,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:39:23,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:41194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34050 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:40:06,181 - lava_callback - INFO - Received callback for node 69af9202275c8d3b7939ae7d INFO: 172.20.0.1:34066 - "POST /node/69af9202275c8d3b7939ae7d HTTP/1.0" 202 Accepted 2026-03-10 03:40:06,187 - lava_callback - INFO - Starting async job submit for node 69af9202275c8d3b7939ae7d 2026-03-10 03:40:06,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:40:06,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af9202275c8d3b7939ae7d HTTP/1.1" 200 None 2026-03-10 03:40:06,260 - lava_callback - DEBUG - Processing node 69af9202275c8d3b7939ae7d: baseline-arm64-broonie 2026-03-10 03:40:06,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:34070 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:40:06,685 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:40:06,686 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af9202275c8d3b7939ae7d/log.txt.gz 2026-03-10 03:40:06,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:40:07,023 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:40:07,025 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af9202275c8d3b7939ae7d/lava-logs.yaml 2026-03-10 03:40:07,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:40:07,185 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:40:07,187 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af9202275c8d3b7939ae7d/lava_callback.json.gz DEBUG: Setup failed for 69af9202275c8d3b7939ae7d. Transitting job node result: incomplete -> fail 2026-03-10 03:40:07,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:40:07,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af9202275c8d3b7939ae7d HTTP/1.1" 200 None 2026-03-10 03:40:07,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:40:07,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 03:40:07,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:40:07,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af9202275c8d3b7939ae7d HTTP/1.1" 200 None 2026-03-10 03:40:07,398 - lava_callback - INFO - Completed processing callback for node 69af9202275c8d3b7939ae7d INFO: 172.20.0.1:39004 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39016 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:40:23,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:40:23,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:44016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36594 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:40:59,810 - lava_callback - INFO - Received callback for node 69acefc6bcd207e915d78427 INFO: 172.20.0.1:49338 - "POST /node/69acefc6bcd207e915d78427 HTTP/1.0" 202 Accepted 2026-03-10 03:40:59,815 - lava_callback - INFO - Starting async job submit for node 69acefc6bcd207e915d78427 2026-03-10 03:40:59,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:40:59,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acefc6bcd207e915d78427 HTTP/1.1" 200 None 2026-03-10 03:40:59,831 - lava_callback - DEBUG - Processing node 69acefc6bcd207e915d78427: baseline-arm64-baylibre 2026-03-10 03:40:59,831 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acefc6bcd207e915d78427 2026-03-10 03:40:59,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:40:59,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acefc6bcd207e915d78427 HTTP/1.1" 200 None 2026-03-10 03:40:59,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:40:59,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a744905a0c0e10c4819cd5 HTTP/1.1" 200 None 2026-03-10 03:40:59,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:40:59,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acefc6bcd207e915d78427 HTTP/1.1" 200 None 2026-03-10 03:40:59,887 - lava_callback - ERROR - Error processing callback for node 69acefc6bcd207e915d78427: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36012 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43394 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57248 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:42:26,640 - lava_callback - INFO - Received callback for node 69af911c275c8d3b7939ae57 INFO: 172.20.0.1:57260 - "POST /node/69af911c275c8d3b7939ae57 HTTP/1.0" 202 Accepted 2026-03-10 03:42:26,647 - lava_callback - INFO - Starting async job submit for node 69af911c275c8d3b7939ae57 2026-03-10 03:42:26,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:42:26,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af911c275c8d3b7939ae57 HTTP/1.1" 200 None 2026-03-10 03:42:26,696 - lava_callback - DEBUG - Processing node 69af911c275c8d3b7939ae57: baseline-arm64-broonie 2026-03-10 03:42:27,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:42:27,212 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:42:27,214 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af911c275c8d3b7939ae57/log.txt.gz 2026-03-10 03:42:27,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:42:27,672 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:42:27,674 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af911c275c8d3b7939ae57/lava-logs.yaml 2026-03-10 03:42:27,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:42:27,815 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:42:27,817 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af911c275c8d3b7939ae57/lava_callback.json.gz DEBUG: Setup failed for 69af911c275c8d3b7939ae57. Transitting job node result: incomplete -> fail 2026-03-10 03:42:27,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:42:27,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af911c275c8d3b7939ae57 HTTP/1.1" 200 None 2026-03-10 03:42:27,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:42:27,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 03:42:27,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:42:27,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af911c275c8d3b7939ae57 HTTP/1.1" 200 None 2026-03-10 03:42:28,024 - lava_callback - INFO - Completed processing callback for node 69af911c275c8d3b7939ae57 INFO: 172.20.0.1:39376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36020 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:42:53,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:42:53,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:36610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49782 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:43:14,280 - lava_callback - INFO - Received callback for node 69af90c1275c8d3b7939ae44 INFO: 172.20.0.1:37510 - "POST /node/69af90c1275c8d3b7939ae44 HTTP/1.0" 202 Accepted 2026-03-10 03:43:14,287 - lava_callback - INFO - Starting async job submit for node 69af90c1275c8d3b7939ae44 2026-03-10 03:43:14,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:43:14,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af90c1275c8d3b7939ae44 HTTP/1.1" 200 None 2026-03-10 03:43:14,338 - lava_callback - DEBUG - Processing node 69af90c1275c8d3b7939ae44: baseline-nfs-arm64 2026-03-10 03:43:14,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:43:14,756 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:43:14,758 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af90c1275c8d3b7939ae44/log.txt.gz 2026-03-10 03:43:14,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:43:15,152 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:43:15,154 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af90c1275c8d3b7939ae44/lava-logs.yaml 2026-03-10 03:43:15,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:37520 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-10 03:43:15,303 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:43:15,305 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af90c1275c8d3b7939ae44/lava_callback.json.gz 2026-03-10 03:43:15,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:43:15,356 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af90c1275c8d3b7939ae44 HTTP/1.1" 200 None 2026-03-10 03:43:15,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:43:15,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 03:43:15,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:43:15,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af90c1275c8d3b7939ae44 HTTP/1.1" 200 None 2026-03-10 03:43:15,475 - lava_callback - INFO - Completed processing callback for node 69af90c1275c8d3b7939ae44 INFO: 172.20.0.1:37522 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:43:16,694 - lava_callback - INFO - Received callback for node 69af92e4275c8d3b7939aeae INFO: 172.20.0.1:37526 - "POST /node/69af92e4275c8d3b7939aeae HTTP/1.0" 202 Accepted 2026-03-10 03:43:16,700 - lava_callback - INFO - Starting async job submit for node 69af92e4275c8d3b7939aeae 2026-03-10 03:43:16,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:43:16,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af92e4275c8d3b7939aeae HTTP/1.1" 200 None 2026-03-10 03:43:16,752 - lava_callback - DEBUG - Processing node 69af92e4275c8d3b7939aeae: baseline-arm64 2026-03-10 03:43:17,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:43:17,168 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:43:17,170 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af92e4275c8d3b7939aeae/log.txt.gz 2026-03-10 03:43:17,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:43:17,522 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:43:17,524 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af92e4275c8d3b7939aeae/lava-logs.yaml 2026-03-10 03:43:17,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:43:17,700 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:43:17,701 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af92e4275c8d3b7939aeae/lava_callback.json.gz DEBUG: Setup failed for 69af92e4275c8d3b7939aeae. Transitting job node result: incomplete -> fail 2026-03-10 03:43:17,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:43:17,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af92e4275c8d3b7939aeae HTTP/1.1" 200 None 2026-03-10 03:43:17,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:43:17,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 03:43:17,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:43:17,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af92e4275c8d3b7939aeae HTTP/1.1" 200 None 2026-03-10 03:43:17,894 - lava_callback - INFO - Completed processing callback for node 69af92e4275c8d3b7939aeae 2026-03-10 03:43:23,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:43:23,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:38230 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:43:32,544 - lava_callback - INFO - Received callback for node 69af9342275c8d3b7939aec2 INFO: 172.20.0.1:40278 - "POST /node/69af9342275c8d3b7939aec2 HTTP/1.0" 202 Accepted 2026-03-10 03:43:32,550 - lava_callback - INFO - Starting async job submit for node 69af9342275c8d3b7939aec2 2026-03-10 03:43:32,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:43:32,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af9342275c8d3b7939aec2 HTTP/1.1" 200 None 2026-03-10 03:43:32,568 - lava_callback - DEBUG - Processing node 69af9342275c8d3b7939aec2: baseline-arm64-broonie 2026-03-10 03:43:32,573 - lava_callback - WARNING - Failed to upload log 2026-03-10 03:43:32,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:43:32,695 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:43:32,697 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af9342275c8d3b7939aec2/lava-logs.yaml 2026-03-10 03:43:32,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:43:32,819 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:43:32,820 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af9342275c8d3b7939aec2/lava_callback.json.gz 2026-03-10 03:43:32,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:43:32,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af9342275c8d3b7939aec2 HTTP/1.1" 200 None 2026-03-10 03:43:32,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:43:32,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 03:43:32,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:43:32,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af9342275c8d3b7939aec2 HTTP/1.1" 200 None 2026-03-10 03:43:32,888 - lava_callback - INFO - Completed processing callback for node 69af9342275c8d3b7939aec2 INFO: 172.20.0.1:40284 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:43:39,972 - lava_callback - INFO - Received callback for node 69acf067bcd207e915d78459 INFO: 172.20.0.1:57140 - "POST /node/69acf067bcd207e915d78459 HTTP/1.0" 202 Accepted 2026-03-10 03:43:39,978 - lava_callback - INFO - Starting async job submit for node 69acf067bcd207e915d78459 2026-03-10 03:43:39,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:43:39,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acf067bcd207e915d78459 HTTP/1.1" 200 None 2026-03-10 03:43:39,994 - lava_callback - DEBUG - Processing node 69acf067bcd207e915d78459: baseline-arm64-baylibre 2026-03-10 03:43:39,994 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acf067bcd207e915d78459 2026-03-10 03:43:39,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:43:40,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acf067bcd207e915d78459 HTTP/1.1" 200 None 2026-03-10 03:43:40,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:43:40,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a744905a0c0e10c4819cd5 HTTP/1.1" 200 None 2026-03-10 03:43:40,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:43:40,044 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acf067bcd207e915d78459 HTTP/1.1" 200 None 2026-03-10 03:43:40,058 - lava_callback - ERROR - Error processing callback for node 69acf067bcd207e915d78459: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57142 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:43:53,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:43:53,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:47402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45830 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53960 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:44:49,011 - lava_callback - INFO - Received callback for node 69af9158275c8d3b7939ae61 INFO: 172.20.0.1:53972 - "POST /node/69af9158275c8d3b7939ae61 HTTP/1.0" 202 Accepted 2026-03-10 03:44:49,017 - lava_callback - INFO - Starting async job submit for node 69af9158275c8d3b7939ae61 2026-03-10 03:44:49,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:44:49,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af9158275c8d3b7939ae61 HTTP/1.1" 200 None 2026-03-10 03:44:49,066 - lava_callback - DEBUG - Processing node 69af9158275c8d3b7939ae61: baseline-arm64 2026-03-10 03:44:49,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:44:49,501 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:44:49,503 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af9158275c8d3b7939ae61/log.txt.gz 2026-03-10 03:44:49,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:44:49,905 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:44:49,908 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af9158275c8d3b7939ae61/lava-logs.yaml 2026-03-10 03:44:49,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:44:50,044 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:44:50,046 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af9158275c8d3b7939ae61/lava_callback.json.gz 2026-03-10 03:44:50,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:44:50,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af9158275c8d3b7939ae61 HTTP/1.1" 200 None 2026-03-10 03:44:50,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:44:50,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 03:44:50,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:44:50,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af9158275c8d3b7939ae61 HTTP/1.1" 200 None 2026-03-10 03:44:50,208 - lava_callback - INFO - Completed processing callback for node 69af9158275c8d3b7939ae61 2026-03-10 03:44:53,971 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:44:53,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:40198 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:45:00,076 - lava_callback - INFO - Received callback for node 69acf0b7bcd207e915d7847c INFO: 172.20.0.1:49378 - "POST /node/69acf0b7bcd207e915d7847c HTTP/1.0" 202 Accepted 2026-03-10 03:45:00,081 - lava_callback - INFO - Starting async job submit for node 69acf0b7bcd207e915d7847c 2026-03-10 03:45:00,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:45:00,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acf0b7bcd207e915d7847c HTTP/1.1" 200 None 2026-03-10 03:45:00,121 - lava_callback - DEBUG - Processing node 69acf0b7bcd207e915d7847c: baseline-arm64-baylibre 2026-03-10 03:45:00,121 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acf0b7bcd207e915d7847c 2026-03-10 03:45:00,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:45:00,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acf0b7bcd207e915d7847c HTTP/1.1" 200 None 2026-03-10 03:45:00,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:45:00,157 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a744905a0c0e10c4819cd5 HTTP/1.1" 200 None 2026-03-10 03:45:00,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:45:00,183 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acf0b7bcd207e915d7847c HTTP/1.1" 200 None 2026-03-10 03:45:00,184 - lava_callback - ERROR - Error processing callback for node 69acf0b7bcd207e915d7847c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49722 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47348 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39566 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:46:57,271 - lava_callback - INFO - Received callback for node 69af93cb275c8d3b7939aedf INFO: 172.20.0.1:39578 - "POST /node/69af93cb275c8d3b7939aedf HTTP/1.0" 202 Accepted 2026-03-10 03:46:57,277 - lava_callback - INFO - Starting async job submit for node 69af93cb275c8d3b7939aedf 2026-03-10 03:46:57,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:46:57,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af93cb275c8d3b7939aedf HTTP/1.1" 200 None 2026-03-10 03:46:57,339 - lava_callback - DEBUG - Processing node 69af93cb275c8d3b7939aedf: baseline-arm64 2026-03-10 03:46:57,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:46:57,878 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:46:57,880 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af93cb275c8d3b7939aedf/log.txt.gz 2026-03-10 03:46:58,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:46:58,330 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:46:58,332 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af93cb275c8d3b7939aedf/lava-logs.yaml 2026-03-10 03:46:58,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:46:58,479 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:46:58,481 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af93cb275c8d3b7939aedf/lava_callback.json.gz DEBUG: Setup failed for 69af93cb275c8d3b7939aedf. Transitting job node result: incomplete -> fail 2026-03-10 03:46:58,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:46:58,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af93cb275c8d3b7939aedf HTTP/1.1" 200 None 2026-03-10 03:46:58,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:46:58,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 03:46:58,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:46:58,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af93cb275c8d3b7939aedf HTTP/1.1" 200 None 2026-03-10 03:46:58,700 - lava_callback - INFO - Completed processing callback for node 69af93cb275c8d3b7939aedf INFO: 172.20.0.1:60780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35478 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35480 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:47:23,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:47:24,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:36912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52750 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:47:40,437 - lava_callback - INFO - Received callback for node 69acf15abcd207e915d784b1 INFO: 172.20.0.1:60390 - "POST /node/69acf15abcd207e915d784b1 HTTP/1.0" 202 Accepted 2026-03-10 03:47:40,443 - lava_callback - INFO - Starting async job submit for node 69acf15abcd207e915d784b1 2026-03-10 03:47:40,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:47:40,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acf15abcd207e915d784b1 HTTP/1.1" 200 None 2026-03-10 03:47:40,459 - lava_callback - DEBUG - Processing node 69acf15abcd207e915d784b1: baseline-arm64-baylibre 2026-03-10 03:47:40,459 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acf15abcd207e915d784b1 2026-03-10 03:47:40,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:47:40,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acf15abcd207e915d784b1 HTTP/1.1" 200 None 2026-03-10 03:47:40,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:47:40,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a744905a0c0e10c4819cd5 HTTP/1.1" 200 None 2026-03-10 03:47:40,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:47:40,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acf15abcd207e915d784b1 HTTP/1.1" 200 None 2026-03-10 03:47:40,515 - lava_callback - ERROR - Error processing callback for node 69acf15abcd207e915d784b1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33476 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:48:12,672 - lava_callback - INFO - Received callback for node 69af941a275c8d3b7939aeed INFO: 172.20.0.1:34648 - "POST /node/69af941a275c8d3b7939aeed HTTP/1.0" 202 Accepted 2026-03-10 03:48:12,678 - lava_callback - INFO - Starting async job submit for node 69af941a275c8d3b7939aeed 2026-03-10 03:48:12,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:48:12,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af941a275c8d3b7939aeed HTTP/1.1" 200 None 2026-03-10 03:48:12,724 - lava_callback - DEBUG - Processing node 69af941a275c8d3b7939aeed: baseline-arm64 2026-03-10 03:48:13,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:48:13,256 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:48:13,258 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af941a275c8d3b7939aeed/log.txt.gz 2026-03-10 03:48:13,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:48:13,694 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:48:13,696 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af941a275c8d3b7939aeed/lava-logs.yaml 2026-03-10 03:48:13,704 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:48:13,866 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:48:13,868 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af941a275c8d3b7939aeed/lava_callback.json.gz DEBUG: Setup failed for 69af941a275c8d3b7939aeed. Transitting job node result: incomplete -> fail 2026-03-10 03:48:13,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:48:13,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af941a275c8d3b7939aeed HTTP/1.1" 200 None 2026-03-10 03:48:13,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:48:13,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 03:48:13,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:48:13,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af941a275c8d3b7939aeed HTTP/1.1" 200 None 2026-03-10 03:48:14,057 - lava_callback - INFO - Completed processing callback for node 69af941a275c8d3b7939aeed INFO: 172.20.0.1:34664 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34668 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:48:24,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:48:24,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:36922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60456 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60472 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:49:20,306 - lava_callback - INFO - Received callback for node 69acf1bdbcd207e915d784d2 INFO: 172.20.0.1:51416 - "POST /node/69acf1bdbcd207e915d784d2 HTTP/1.0" 202 Accepted 2026-03-10 03:49:20,312 - lava_callback - INFO - Starting async job submit for node 69acf1bdbcd207e915d784d2 2026-03-10 03:49:20,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:49:20,327 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acf1bdbcd207e915d784d2 HTTP/1.1" 200 None 2026-03-10 03:49:20,328 - lava_callback - DEBUG - Processing node 69acf1bdbcd207e915d784d2: baseline-arm64-baylibre 2026-03-10 03:49:20,328 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acf1bdbcd207e915d784d2 2026-03-10 03:49:20,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:49:20,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acf1bdbcd207e915d784d2 HTTP/1.1" 200 None 2026-03-10 03:49:20,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:49:20,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a744905a0c0e10c4819cd5 HTTP/1.1" 200 None 2026-03-10 03:49:20,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:49:20,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acf1bdbcd207e915d784d2 HTTP/1.1" 200 None 2026-03-10 03:49:20,386 - lava_callback - ERROR - Error processing callback for node 69acf1bdbcd207e915d784d2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39262 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45104 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:50:31,271 - lava_callback - INFO - Received callback for node 69af929a275c8d3b7939aea0 INFO: 172.20.0.1:37682 - "POST /node/69af929a275c8d3b7939aea0 HTTP/1.0" 202 Accepted 2026-03-10 03:50:31,278 - lava_callback - INFO - Starting async job submit for node 69af929a275c8d3b7939aea0 2026-03-10 03:50:31,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:50:31,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af929a275c8d3b7939aea0 HTTP/1.1" 200 None 2026-03-10 03:50:31,320 - lava_callback - DEBUG - Processing node 69af929a275c8d3b7939aea0: baseline-nfs-arm64 2026-03-10 03:50:31,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:50:32,003 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:50:32,005 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af929a275c8d3b7939aea0/log.txt.gz 2026-03-10 03:50:32,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:50:32,468 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:50:32,471 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af929a275c8d3b7939aea0/lava-logs.yaml 2026-03-10 03:50:32,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:50:32,638 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:50:32,640 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af929a275c8d3b7939aea0/lava_callback.json.gz 2026-03-10 03:50:32,669 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:50:32,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af929a275c8d3b7939aea0 HTTP/1.1" 200 None 2026-03-10 03:50:32,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:50:32,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 03:50:32,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:50:32,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af929a275c8d3b7939aea0 HTTP/1.1" 200 None 2026-03-10 03:50:32,817 - lava_callback - INFO - Completed processing callback for node 69af929a275c8d3b7939aea0 2026-03-10 03:50:36,454 - lava_callback - INFO - Received callback for node 69af949b275c8d3b7939af03 INFO: 172.20.0.1:37694 - "POST /node/69af949b275c8d3b7939af03 HTTP/1.0" 202 Accepted 2026-03-10 03:50:36,460 - lava_callback - INFO - Starting async job submit for node 69af949b275c8d3b7939af03 2026-03-10 03:50:36,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:50:36,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af949b275c8d3b7939af03 HTTP/1.1" 200 None 2026-03-10 03:50:36,508 - lava_callback - DEBUG - Processing node 69af949b275c8d3b7939af03: baseline-x86 INFO: 172.20.0.1:37710 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:50:36,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:50:36,890 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:50:36,891 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69af949b275c8d3b7939af03/log.txt.gz 2026-03-10 03:50:37,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:50:37,248 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:50:37,251 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69af949b275c8d3b7939af03/lava-logs.yaml 2026-03-10 03:50:37,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:50:37,464 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:50:37,466 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69af949b275c8d3b7939af03/lava_callback.json.gz 2026-03-10 03:50:37,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:50:37,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af949b275c8d3b7939af03 HTTP/1.1" 200 None 2026-03-10 03:50:37,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:50:37,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af635a275c8d3b79399ddd HTTP/1.1" 200 None 2026-03-10 03:50:37,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:50:37,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af949b275c8d3b7939af03 HTTP/1.1" 200 None 2026-03-10 03:50:37,554 - lava_callback - INFO - Completed processing callback for node 69af949b275c8d3b7939af03 2026-03-10 03:50:40,330 - lava_callback - INFO - Received callback for node 69acf20ebcd207e915d784ef INFO: 172.20.0.1:50792 - "POST /node/69acf20ebcd207e915d784ef HTTP/1.0" 202 Accepted 2026-03-10 03:50:40,336 - lava_callback - INFO - Starting async job submit for node 69acf20ebcd207e915d784ef 2026-03-10 03:50:40,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:50:40,351 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acf20ebcd207e915d784ef HTTP/1.1" 200 None 2026-03-10 03:50:40,352 - lava_callback - DEBUG - Processing node 69acf20ebcd207e915d784ef: baseline-arm64-baylibre 2026-03-10 03:50:40,352 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acf20ebcd207e915d784ef 2026-03-10 03:50:40,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:50:40,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acf20ebcd207e915d784ef HTTP/1.1" 200 None 2026-03-10 03:50:40,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:50:40,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a744905a0c0e10c4819cd5 HTTP/1.1" 200 None 2026-03-10 03:50:40,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:50:40,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acf20ebcd207e915d784ef HTTP/1.1" 200 None 2026-03-10 03:50:40,413 - lava_callback - ERROR - Error processing callback for node 69acf20ebcd207e915d784ef: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50802 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:50:54,029 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:50:54,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:52428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41698 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:51:06,044 - lava_callback - INFO - Received callback for node 69af94d0275c8d3b7939af15 INFO: 172.20.0.1:41702 - "POST /node/69af94d0275c8d3b7939af15 HTTP/1.0" 202 Accepted 2026-03-10 03:51:06,051 - lava_callback - INFO - Starting async job submit for node 69af94d0275c8d3b7939af15 2026-03-10 03:51:06,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:51:06,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af94d0275c8d3b7939af15 HTTP/1.1" 200 None 2026-03-10 03:51:06,099 - lava_callback - DEBUG - Processing node 69af94d0275c8d3b7939af15: baseline-nfs-arm64 INFO: 172.20.0.1:41704 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:51:06,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:51:06,648 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:51:06,650 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af94d0275c8d3b7939af15/log.txt.gz 2026-03-10 03:51:06,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:51:07,107 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:51:07,109 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af94d0275c8d3b7939af15/lava-logs.yaml 2026-03-10 03:51:07,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:51:07,255 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:51:07,257 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af94d0275c8d3b7939af15/lava_callback.json.gz DEBUG: Setup failed for 69af94d0275c8d3b7939af15. Transitting job node result: incomplete -> fail 2026-03-10 03:51:07,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:51:07,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af94d0275c8d3b7939af15 HTTP/1.1" 200 None 2026-03-10 03:51:07,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:51:07,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 03:51:07,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:51:07,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af94d0275c8d3b7939af15 HTTP/1.1" 200 None 2026-03-10 03:51:07,454 - lava_callback - INFO - Completed processing callback for node 69af94d0275c8d3b7939af15 INFO: 172.20.0.1:35070 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35086 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:51:24,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:51:24,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:54332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33708 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35448 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:53:02,852 - lava_callback - INFO - Received callback for node 69af951d275c8d3b7939af29 INFO: 172.20.0.1:50102 - "POST /node/69af951d275c8d3b7939af29 HTTP/1.0" 202 Accepted 2026-03-10 03:53:02,859 - lava_callback - INFO - Starting async job submit for node 69af951d275c8d3b7939af29 2026-03-10 03:53:02,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:53:02,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af951d275c8d3b7939af29 HTTP/1.1" 200 None 2026-03-10 03:53:02,905 - lava_callback - DEBUG - Processing node 69af951d275c8d3b7939af29: baseline-nfs-arm64 2026-03-10 03:53:03,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:53:03,473 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:53:03,475 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af951d275c8d3b7939af29/log.txt.gz 2026-03-10 03:53:03,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:53:03,991 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:53:03,993 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af951d275c8d3b7939af29/lava-logs.yaml 2026-03-10 03:53:04,001 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:53:04,158 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:53:04,160 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af951d275c8d3b7939af29/lava_callback.json.gz DEBUG: Setup failed for 69af951d275c8d3b7939af29. Transitting job node result: incomplete -> fail 2026-03-10 03:53:04,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:53:04,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af951d275c8d3b7939af29 HTTP/1.1" 200 None 2026-03-10 03:53:04,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:53:04,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 03:53:04,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:53:04,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af951d275c8d3b7939af29 HTTP/1.1" 200 None 2026-03-10 03:53:04,376 - lava_callback - INFO - Completed processing callback for node 69af951d275c8d3b7939af29 INFO: 172.20.0.1:50114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35944 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35952 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:53:24,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:53:24,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:42246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60550 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:53:40,546 - lava_callback - INFO - Received callback for node 69acf2b5bcd207e915d7852a INFO: 172.20.0.1:35334 - "POST /node/69acf2b5bcd207e915d7852a HTTP/1.0" 202 Accepted 2026-03-10 03:53:40,551 - lava_callback - INFO - Starting async job submit for node 69acf2b5bcd207e915d7852a 2026-03-10 03:53:40,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:53:40,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acf2b5bcd207e915d7852a HTTP/1.1" 200 None 2026-03-10 03:53:40,568 - lava_callback - DEBUG - Processing node 69acf2b5bcd207e915d7852a: baseline-arm-baylibre 2026-03-10 03:53:40,568 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acf2b5bcd207e915d7852a 2026-03-10 03:53:40,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:53:40,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acf2b5bcd207e915d7852a HTTP/1.1" 200 None 2026-03-10 03:53:40,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:53:40,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a7447b5a0c0e10c4819cc3 HTTP/1.1" 200 None 2026-03-10 03:53:40,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:53:40,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acf2b5bcd207e915d7852a HTTP/1.1" 200 None 2026-03-10 03:53:40,626 - lava_callback - ERROR - Error processing callback for node 69acf2b5bcd207e915d7852a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-10 03:53:45,132 - lava_callback - INFO - Received callback for node 69af9576275c8d3b7939af39 INFO: 172.20.0.1:35346 - "POST /node/69af9576275c8d3b7939af39 HTTP/1.0" 202 Accepted 2026-03-10 03:53:45,138 - lava_callback - INFO - Starting async job submit for node 69af9576275c8d3b7939af39 2026-03-10 03:53:45,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:53:45,183 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af9576275c8d3b7939af39 HTTP/1.1" 200 None 2026-03-10 03:53:45,184 - lava_callback - DEBUG - Processing node 69af9576275c8d3b7939af39: baseline-arm64-foundriesio 2026-03-10 03:53:45,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:53:45,535 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:53:45,537 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69af9576275c8d3b7939af39/log.txt.gz 2026-03-10 03:53:45,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:53:45,839 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:53:45,841 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69af9576275c8d3b7939af39/lava-logs.yaml 2026-03-10 03:53:45,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:53:45,994 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:53:45,996 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69af9576275c8d3b7939af39/lava_callback.json.gz DEBUG: Setup failed for 69af9576275c8d3b7939af39. Transitting job node result: incomplete -> fail 2026-03-10 03:53:46,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:53:46,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af9576275c8d3b7939af39 HTTP/1.1" 200 None 2026-03-10 03:53:46,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:53:46,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 03:53:46,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:53:46,129 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af9576275c8d3b7939af39 HTTP/1.1" 200 None 2026-03-10 03:53:46,199 - lava_callback - INFO - Completed processing callback for node 69af9576275c8d3b7939af39 INFO: 172.20.0.1:35362 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:53:54,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:53:54,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:33894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55960 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55964 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:54:20,477 - lava_callback - INFO - Received callback for node 69acf2ddbcd207e915d78544 INFO: 172.20.0.1:43952 - "POST /node/69acf2ddbcd207e915d78544 HTTP/1.0" 202 Accepted 2026-03-10 03:54:20,482 - lava_callback - INFO - Starting async job submit for node 69acf2ddbcd207e915d78544 2026-03-10 03:54:20,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:54:20,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acf2ddbcd207e915d78544 HTTP/1.1" 200 None 2026-03-10 03:54:20,499 - lava_callback - DEBUG - Processing node 69acf2ddbcd207e915d78544: baseline-arm-baylibre 2026-03-10 03:54:20,499 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acf2ddbcd207e915d78544 2026-03-10 03:54:20,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:54:20,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acf2ddbcd207e915d78544 HTTP/1.1" 200 None 2026-03-10 03:54:20,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:54:20,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a7447b5a0c0e10c4819cc3 HTTP/1.1" 200 None 2026-03-10 03:54:20,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:54:20,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acf2ddbcd207e915d78544 HTTP/1.1" 200 None 2026-03-10 03:54:20,569 - lava_callback - ERROR - Error processing callback for node 69acf2ddbcd207e915d78544: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43960 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:54:28,667 - lava_callback - INFO - Received callback for node 69af937d275c8d3b7939aed2 INFO: 172.20.0.1:43970 - "POST /node/69af937d275c8d3b7939aed2 HTTP/1.0" 202 Accepted 2026-03-10 03:54:28,674 - lava_callback - INFO - Starting async job submit for node 69af937d275c8d3b7939aed2 2026-03-10 03:54:28,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:54:28,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af937d275c8d3b7939aed2 HTTP/1.1" 200 None 2026-03-10 03:54:28,714 - lava_callback - DEBUG - Processing node 69af937d275c8d3b7939aed2: baseline-arm64 2026-03-10 03:54:29,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:54:29,463 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:54:29,465 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af937d275c8d3b7939aed2/log.txt.gz 2026-03-10 03:54:29,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:54:29,983 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:54:29,985 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af937d275c8d3b7939aed2/lava-logs.yaml 2026-03-10 03:54:29,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:54:30,197 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:54:30,199 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af937d275c8d3b7939aed2/lava_callback.json.gz 2026-03-10 03:54:30,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:54:30,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af937d275c8d3b7939aed2 HTTP/1.1" 200 None 2026-03-10 03:54:30,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:54:30,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 03:54:30,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:54:30,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af937d275c8d3b7939aed2 HTTP/1.1" 200 None 2026-03-10 03:54:30,424 - lava_callback - INFO - Completed processing callback for node 69af937d275c8d3b7939aed2 INFO: 172.20.0.1:39722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42816 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:54:54,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:54:54,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:50554 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:55:00,581 - lava_callback - INFO - Received callback for node 69acf306bcd207e915d7855b INFO: 172.20.0.1:40192 - "POST /node/69acf306bcd207e915d7855b HTTP/1.0" 202 Accepted 2026-03-10 03:55:00,586 - lava_callback - INFO - Starting async job submit for node 69acf306bcd207e915d7855b 2026-03-10 03:55:00,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:55:00,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acf306bcd207e915d7855b HTTP/1.1" 200 None 2026-03-10 03:55:00,602 - lava_callback - DEBUG - Processing node 69acf306bcd207e915d7855b: baseline-arm-baylibre 2026-03-10 03:55:00,602 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acf306bcd207e915d7855b 2026-03-10 03:55:00,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:55:00,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acf306bcd207e915d7855b HTTP/1.1" 200 None 2026-03-10 03:55:00,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:55:00,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a7447b5a0c0e10c4819cc3 HTTP/1.1" 200 None 2026-03-10 03:55:00,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:55:00,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acf306bcd207e915d7855b HTTP/1.1" 200 None 2026-03-10 03:55:00,659 - lava_callback - ERROR - Error processing callback for node 69acf306bcd207e915d7855b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51654 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36360 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:56:00,548 - lava_callback - INFO - Received callback for node 69acf342bcd207e915d78581 INFO: 172.20.0.1:58552 - "POST /node/69acf342bcd207e915d78581 HTTP/1.0" 202 Accepted 2026-03-10 03:56:00,555 - lava_callback - INFO - Starting async job submit for node 69acf342bcd207e915d78581 2026-03-10 03:56:00,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:56:00,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acf342bcd207e915d78581 HTTP/1.1" 200 None 2026-03-10 03:56:00,573 - lava_callback - DEBUG - Processing node 69acf342bcd207e915d78581: baseline-arm-baylibre 2026-03-10 03:56:00,573 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acf342bcd207e915d78581 2026-03-10 03:56:00,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:56:00,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acf342bcd207e915d78581 HTTP/1.1" 200 None 2026-03-10 03:56:00,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:56:00,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a7447b5a0c0e10c4819cc3 HTTP/1.1" 200 None 2026-03-10 03:56:00,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:56:00,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acf342bcd207e915d78581 HTTP/1.1" 200 None 2026-03-10 03:56:00,629 - lava_callback - ERROR - Error processing callback for node 69acf342bcd207e915d78581: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52976 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47884 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:56:40,649 - lava_callback - INFO - Received callback for node 69acf36cbcd207e915d7859a INFO: 172.20.0.1:46294 - "POST /node/69acf36cbcd207e915d7859a HTTP/1.0" 202 Accepted 2026-03-10 03:56:40,655 - lava_callback - INFO - Starting async job submit for node 69acf36cbcd207e915d7859a 2026-03-10 03:56:40,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:56:40,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acf36cbcd207e915d7859a HTTP/1.1" 200 None 2026-03-10 03:56:40,670 - lava_callback - DEBUG - Processing node 69acf36cbcd207e915d7859a: baseline-arm-baylibre 2026-03-10 03:56:40,670 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acf36cbcd207e915d7859a 2026-03-10 03:56:40,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:56:40,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acf36cbcd207e915d7859a HTTP/1.1" 200 None 2026-03-10 03:56:40,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:56:40,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a7447b5a0c0e10c4819cc3 HTTP/1.1" 200 None 2026-03-10 03:56:40,704 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:56:40,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acf36cbcd207e915d7859a HTTP/1.1" 200 None 2026-03-10 03:56:40,728 - lava_callback - ERROR - Error processing callback for node 69acf36cbcd207e915d7859a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48188 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48204 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:57:20,649 - lava_callback - INFO - Received callback for node 69acf394bcd207e915d785ac INFO: 172.20.0.1:56640 - "POST /node/69acf394bcd207e915d785ac HTTP/1.0" 202 Accepted 2026-03-10 03:57:20,654 - lava_callback - INFO - Starting async job submit for node 69acf394bcd207e915d785ac 2026-03-10 03:57:20,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:57:20,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acf394bcd207e915d785ac HTTP/1.1" 200 None 2026-03-10 03:57:20,670 - lava_callback - DEBUG - Processing node 69acf394bcd207e915d785ac: baseline-arm-baylibre 2026-03-10 03:57:20,671 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acf394bcd207e915d785ac 2026-03-10 03:57:20,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:57:20,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acf394bcd207e915d785ac HTTP/1.1" 200 None 2026-03-10 03:57:20,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:57:20,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a7447b5a0c0e10c4819cc3 HTTP/1.1" 200 None 2026-03-10 03:57:20,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:57:20,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acf394bcd207e915d785ac HTTP/1.1" 200 None 2026-03-10 03:57:20,738 - lava_callback - ERROR - Error processing callback for node 69acf394bcd207e915d785ac: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60076 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40064 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37550 - "GET / HTTP/1.0" 200 OK 2026-03-10 03:59:52,415 - lava_callback - INFO - Received callback for node 69af971c275c8d3b7939af8d INFO: 172.20.0.1:40420 - "POST /node/69af971c275c8d3b7939af8d HTTP/1.0" 202 Accepted 2026-03-10 03:59:52,420 - lava_callback - INFO - Starting async job submit for node 69af971c275c8d3b7939af8d 2026-03-10 03:59:52,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:59:52,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af971c275c8d3b7939af8d HTTP/1.1" 200 None 2026-03-10 03:59:52,438 - lava_callback - DEBUG - Processing node 69af971c275c8d3b7939af8d: baseline-arm64 2026-03-10 03:59:52,444 - lava_callback - WARNING - Failed to upload log 2026-03-10 03:59:52,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:59:52,616 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:59:52,618 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af971c275c8d3b7939af8d/lava-logs.yaml 2026-03-10 03:59:52,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 03:59:52,772 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 03:59:52,774 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af971c275c8d3b7939af8d/lava_callback.json.gz 2026-03-10 03:59:52,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:59:52,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af971c275c8d3b7939af8d HTTP/1.1" 200 None 2026-03-10 03:59:52,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:59:52,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 03:59:52,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:59:52,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af971c275c8d3b7939af8d HTTP/1.1" 200 None 2026-03-10 03:59:52,855 - lava_callback - INFO - Completed processing callback for node 69af971c275c8d3b7939af8d 2026-03-10 03:59:54,130 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 03:59:54,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:40422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59288 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42298 - "GET / HTTP/1.0" 200 OK 2026-03-10 04:01:01,616 - lava_callback - INFO - Received callback for node 69af96cc275c8d3b7939af7d INFO: 172.20.0.1:46774 - "POST /node/69af96cc275c8d3b7939af7d HTTP/1.0" 202 Accepted 2026-03-10 04:01:01,622 - lava_callback - INFO - Starting async job submit for node 69af96cc275c8d3b7939af7d 2026-03-10 04:01:01,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:01:01,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af96cc275c8d3b7939af7d HTTP/1.1" 200 None 2026-03-10 04:01:01,676 - lava_callback - DEBUG - Processing node 69af96cc275c8d3b7939af7d: baseline-arm64 2026-03-10 04:01:02,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 04:01:02,243 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 04:01:02,244 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af96cc275c8d3b7939af7d/log.txt.gz 2026-03-10 04:01:02,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 04:01:02,722 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 04:01:02,724 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af96cc275c8d3b7939af7d/lava-logs.yaml 2026-03-10 04:01:02,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 04:01:02,874 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 04:01:02,876 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af96cc275c8d3b7939af7d/lava_callback.json.gz DEBUG: Setup failed for 69af96cc275c8d3b7939af7d. Transitting job node result: incomplete -> fail 2026-03-10 04:01:02,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:01:02,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af96cc275c8d3b7939af7d HTTP/1.1" 200 None 2026-03-10 04:01:02,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:01:02,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 04:01:02,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:01:03,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af96cc275c8d3b7939af7d HTTP/1.1" 200 None 2026-03-10 04:01:03,089 - lava_callback - INFO - Completed processing callback for node 69af96cc275c8d3b7939af7d INFO: 172.20.0.1:46790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35440 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35450 - "GET / HTTP/1.0" 200 OK 2026-03-10 04:01:24,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:01:24,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:41816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37580 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37584 - "GET / HTTP/1.0" 200 OK 2026-03-10 04:02:18,396 - lava_callback - INFO - Received callback for node 69af97ba275c8d3b7939afa4 INFO: 172.20.0.1:37596 - "POST /node/69af97ba275c8d3b7939afa4 HTTP/1.0" 202 Accepted 2026-03-10 04:02:18,401 - lava_callback - INFO - Starting async job submit for node 69af97ba275c8d3b7939afa4 2026-03-10 04:02:18,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:02:18,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af97ba275c8d3b7939afa4 HTTP/1.1" 200 None 2026-03-10 04:02:18,420 - lava_callback - DEBUG - Processing node 69af97ba275c8d3b7939afa4: baseline-nfs-arm64 2026-03-10 04:02:18,427 - lava_callback - WARNING - Failed to upload log 2026-03-10 04:02:18,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 04:02:22,997 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 04:02:22,999 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af97ba275c8d3b7939afa4/lava-logs.yaml 2026-03-10 04:02:23,000 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 04:02:23,148 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 04:02:23,150 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af97ba275c8d3b7939afa4/lava_callback.json.gz 2026-03-10 04:02:23,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:02:23,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af97ba275c8d3b7939afa4 HTTP/1.1" 200 None 2026-03-10 04:02:23,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:02:23,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 04:02:23,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:02:23,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af97ba275c8d3b7939afa4 HTTP/1.1" 200 None 2026-03-10 04:02:23,215 - lava_callback - INFO - Completed processing callback for node 69af97ba275c8d3b7939afa4 2026-03-10 04:02:24,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:02:24,185 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:33000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42506 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58680 - "GET / HTTP/1.0" 200 OK 2026-03-10 04:03:57,295 - lava_callback - INFO - Received callback for node 69af976b275c8d3b7939af95 INFO: 172.20.0.1:58692 - "POST /node/69af976b275c8d3b7939af95 HTTP/1.0" 202 Accepted 2026-03-10 04:03:57,301 - lava_callback - INFO - Starting async job submit for node 69af976b275c8d3b7939af95 2026-03-10 04:03:57,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:03:57,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af976b275c8d3b7939af95 HTTP/1.1" 200 None 2026-03-10 04:03:57,349 - lava_callback - DEBUG - Processing node 69af976b275c8d3b7939af95: baseline-nfs-arm64 2026-03-10 04:03:57,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 04:03:57,940 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 04:03:57,942 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af976b275c8d3b7939af95/log.txt.gz 2026-03-10 04:03:58,171 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 04:03:58,419 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 04:03:58,421 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af976b275c8d3b7939af95/lava-logs.yaml 2026-03-10 04:03:58,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 04:03:58,573 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 04:03:58,575 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af976b275c8d3b7939af95/lava_callback.json.gz DEBUG: Setup failed for 69af976b275c8d3b7939af95. Transitting job node result: incomplete -> fail 2026-03-10 04:03:58,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:03:58,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af976b275c8d3b7939af95 HTTP/1.1" 200 None 2026-03-10 04:03:58,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:03:58,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 04:03:58,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:03:58,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af976b275c8d3b7939af95 HTTP/1.1" 200 None 2026-03-10 04:03:58,800 - lava_callback - INFO - Completed processing callback for node 69af976b275c8d3b7939af95 INFO: 172.20.0.1:42966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42972 - "GET / HTTP/1.0" 200 OK 2026-03-10 04:04:07,241 - lava_callback - INFO - Received callback for node 69af9413275c8d3b7939aeea INFO: 172.20.0.1:42982 - "POST /node/69af9413275c8d3b7939aeea HTTP/1.0" 202 Accepted 2026-03-10 04:04:07,248 - lava_callback - INFO - Starting async job submit for node 69af9413275c8d3b7939aeea 2026-03-10 04:04:07,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:04:07,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af9413275c8d3b7939aeea HTTP/1.1" 200 None 2026-03-10 04:04:07,320 - lava_callback - DEBUG - Processing node 69af9413275c8d3b7939aeea: ltp-smoke 2026-03-10 04:04:07,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 04:04:07,953 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 04:04:07,954 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69af9413275c8d3b7939aeea/log.txt.gz 2026-03-10 04:04:08,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 04:04:08,449 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 04:04:08,452 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69af9413275c8d3b7939aeea/lava-logs.yaml 2026-03-10 04:04:08,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 04:04:08,665 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 04:04:08,667 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69af9413275c8d3b7939aeea/lava_callback.json.gz 2026-03-10 04:04:08,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:04:08,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af9413275c8d3b7939aeea HTTP/1.1" 200 None 2026-03-10 04:04:08,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:04:08,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6334275c8d3b79399dba HTTP/1.1" 200 None 2026-03-10 04:04:08,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:04:08,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af9413275c8d3b7939aeea HTTP/1.1" 200 None 2026-03-10 04:04:08,988 - lava_callback - INFO - Completed processing callback for node 69af9413275c8d3b7939aeea INFO: 172.20.0.1:48186 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48198 - "GET / HTTP/1.0" 200 OK 2026-03-10 04:04:24,187 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:04:24,204 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:38966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33944 - "GET / HTTP/1.0" 200 OK 2026-03-10 04:04:51,583 - lava_callback - INFO - Received callback for node 69af9804275c8d3b7939afad INFO: 172.20.0.1:43972 - "POST /node/69af9804275c8d3b7939afad HTTP/1.0" 202 Accepted 2026-03-10 04:04:51,591 - lava_callback - INFO - Starting async job submit for node 69af9804275c8d3b7939afad 2026-03-10 04:04:51,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:04:51,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af9804275c8d3b7939afad HTTP/1.1" 200 None 2026-03-10 04:04:51,637 - lava_callback - DEBUG - Processing node 69af9804275c8d3b7939afad: baseline-arm64 2026-03-10 04:04:52,187 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 04:04:52,313 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 04:04:52,315 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af9804275c8d3b7939afad/log.txt.gz 2026-03-10 04:04:52,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 04:04:52,847 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 04:04:52,849 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af9804275c8d3b7939afad/lava-logs.yaml 2026-03-10 04:04:52,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 04:04:53,051 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 04:04:53,053 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af9804275c8d3b7939afad/lava_callback.json.gz DEBUG: Setup failed for 69af9804275c8d3b7939afad. Transitting job node result: incomplete -> fail 2026-03-10 04:04:53,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:04:53,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af9804275c8d3b7939afad HTTP/1.1" 200 None 2026-03-10 04:04:53,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:04:53,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 04:04:53,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:04:53,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af9804275c8d3b7939afad HTTP/1.1" 200 None 2026-03-10 04:04:53,258 - lava_callback - INFO - Completed processing callback for node 69af9804275c8d3b7939afad 2026-03-10 04:04:54,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:04:54,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:43974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36068 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54108 - "GET / HTTP/1.0" 200 OK 2026-03-10 04:06:11,373 - lava_callback - INFO - Received callback for node 69af949e275c8d3b7939af06 INFO: 172.20.0.1:37514 - "POST /node/69af949e275c8d3b7939af06 HTTP/1.0" 202 Accepted 2026-03-10 04:06:11,380 - lava_callback - INFO - Starting async job submit for node 69af949e275c8d3b7939af06 2026-03-10 04:06:11,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:06:11,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af949e275c8d3b7939af06 HTTP/1.1" 200 None 2026-03-10 04:06:11,452 - lava_callback - DEBUG - Processing node 69af949e275c8d3b7939af06: baseline-arm 2026-03-10 04:06:12,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 04:06:12,231 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 04:06:12,233 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69af949e275c8d3b7939af06/log.txt.gz 2026-03-10 04:06:12,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 04:06:12,843 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 04:06:12,845 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69af949e275c8d3b7939af06/lava-logs.yaml 2026-03-10 04:06:12,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 04:06:13,043 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 04:06:13,045 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69af949e275c8d3b7939af06/lava_callback.json.gz DEBUG: Setup failed for 69af949e275c8d3b7939af06. Transitting job node result: incomplete -> fail 2026-03-10 04:06:13,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:06:13,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af949e275c8d3b7939af06 HTTP/1.1" 200 None 2026-03-10 04:06:13,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:06:13,190 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af632c275c8d3b79399db4 HTTP/1.1" 200 None 2026-03-10 04:06:13,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:06:13,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af949e275c8d3b7939af06 HTTP/1.1" 200 None 2026-03-10 04:06:13,343 - lava_callback - INFO - Completed processing callback for node 69af949e275c8d3b7939af06 INFO: 172.20.0.1:37516 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37518 - "GET / HTTP/1.0" 200 OK 2026-03-10 04:06:24,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:06:24,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:41150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38442 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51760 - "GET / HTTP/1.0" 200 OK 2026-03-10 04:07:37,848 - lava_callback - INFO - Received callback for node 69af98a5275c8d3b7939afd2 INFO: 172.20.0.1:51766 - "POST /node/69af98a5275c8d3b7939afd2 HTTP/1.0" 202 Accepted 2026-03-10 04:07:37,855 - lava_callback - INFO - Starting async job submit for node 69af98a5275c8d3b7939afd2 2026-03-10 04:07:37,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:07:37,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af98a5275c8d3b7939afd2 HTTP/1.1" 200 None 2026-03-10 04:07:37,917 - lava_callback - DEBUG - Processing node 69af98a5275c8d3b7939afd2: baseline-nfs-arm64 2026-03-10 04:07:38,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 04:07:38,393 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 04:07:38,395 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af98a5275c8d3b7939afd2/log.txt.gz 2026-03-10 04:07:38,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 04:07:38,924 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 04:07:38,926 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af98a5275c8d3b7939afd2/lava-logs.yaml 2026-03-10 04:07:38,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 04:07:39,095 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 04:07:39,097 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af98a5275c8d3b7939afd2/lava_callback.json.gz DEBUG: Setup failed for 69af98a5275c8d3b7939afd2. Transitting job node result: incomplete -> fail 2026-03-10 04:07:39,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:07:39,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af98a5275c8d3b7939afd2 HTTP/1.1" 200 None 2026-03-10 04:07:39,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:07:39,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 04:07:39,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:07:39,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af98a5275c8d3b7939afd2 HTTP/1.1" 200 None 2026-03-10 04:07:39,310 - lava_callback - INFO - Completed processing callback for node 69af98a5275c8d3b7939afd2 INFO: 172.20.0.1:52082 - "GET / HTTP/1.0" 200 OK 2026-03-10 04:07:54,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:07:54,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:37656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34402 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51666 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37216 - "GET / HTTP/1.0" 200 OK 2026-03-10 04:10:01,484 - lava_callback - INFO - Received callback for node 69acf697bcd207e915d786bb INFO: 172.20.0.1:55008 - "POST /node/69acf697bcd207e915d786bb HTTP/1.0" 202 Accepted 2026-03-10 04:10:01,490 - lava_callback - INFO - Starting async job submit for node 69acf697bcd207e915d786bb 2026-03-10 04:10:01,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:10:01,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acf697bcd207e915d786bb HTTP/1.1" 200 None 2026-03-10 04:10:01,512 - lava_callback - DEBUG - Processing node 69acf697bcd207e915d786bb: baseline-arm-baylibre 2026-03-10 04:10:01,512 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acf697bcd207e915d786bb 2026-03-10 04:10:01,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:10:01,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acf697bcd207e915d786bb HTTP/1.1" 200 None 2026-03-10 04:10:01,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:10:01,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a750a55a0c0e10c481a794 HTTP/1.1" 200 None 2026-03-10 04:10:01,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:10:01,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acf697bcd207e915d786bb HTTP/1.1" 200 None 2026-03-10 04:10:01,569 - lava_callback - ERROR - Error processing callback for node 69acf697bcd207e915d786bb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33048 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52002 - "GET / HTTP/1.0" 200 OK 2026-03-10 04:10:37,967 - lava_callback - INFO - Received callback for node 69af993e275c8d3b7939aff3 INFO: 172.20.0.1:52010 - "POST /node/69af993e275c8d3b7939aff3 HTTP/1.0" 202 Accepted 2026-03-10 04:10:37,974 - lava_callback - INFO - Starting async job submit for node 69af993e275c8d3b7939aff3 2026-03-10 04:10:38,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:10:38,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af993e275c8d3b7939aff3 HTTP/1.1" 200 None 2026-03-10 04:10:38,022 - lava_callback - DEBUG - Processing node 69af993e275c8d3b7939aff3: baseline-nfs-arm64 2026-03-10 04:10:38,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 04:10:38,727 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 04:10:38,729 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af993e275c8d3b7939aff3/log.txt.gz 2026-03-10 04:10:39,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 04:10:39,373 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 04:10:39,375 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af993e275c8d3b7939aff3/lava-logs.yaml 2026-03-10 04:10:39,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 04:10:39,592 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 04:10:39,594 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af993e275c8d3b7939aff3/lava_callback.json.gz DEBUG: Setup failed for 69af993e275c8d3b7939aff3. Transitting job node result: incomplete -> fail 2026-03-10 04:10:39,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:10:39,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af993e275c8d3b7939aff3 HTTP/1.1" 200 None 2026-03-10 04:10:39,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:10:39,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 04:10:39,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:10:39,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af993e275c8d3b7939aff3 HTTP/1.1" 200 None 2026-03-10 04:10:39,778 - lava_callback - INFO - Completed processing callback for node 69af993e275c8d3b7939aff3 INFO: 172.20.0.1:36412 - "GET / HTTP/1.0" 200 OK 2026-03-10 04:10:54,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:10:54,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:38496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44460 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53430 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48752 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44458 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40660 - "GET / HTTP/1.0" 200 OK 2026-03-10 04:15:12,916 - lava_callback - INFO - Received callback for node 69af9a6b275c8d3b7939b024 INFO: 172.20.0.1:33926 - "POST /node/69af9a6b275c8d3b7939b024 HTTP/1.0" 202 Accepted 2026-03-10 04:15:12,924 - lava_callback - INFO - Starting async job submit for node 69af9a6b275c8d3b7939b024 2026-03-10 04:15:12,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:15:12,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af9a6b275c8d3b7939b024 HTTP/1.1" 200 None 2026-03-10 04:15:12,974 - lava_callback - DEBUG - Processing node 69af9a6b275c8d3b7939b024: baseline-arm64 2026-03-10 04:15:13,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 04:15:13,515 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 04:15:13,516 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af9a6b275c8d3b7939b024/log.txt.gz 2026-03-10 04:15:13,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 04:15:13,979 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 04:15:13,981 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af9a6b275c8d3b7939b024/lava-logs.yaml 2026-03-10 04:15:13,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 04:15:14,159 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 04:15:14,161 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af9a6b275c8d3b7939b024/lava_callback.json.gz DEBUG: Setup failed for 69af9a6b275c8d3b7939b024. Transitting job node result: incomplete -> fail 2026-03-10 04:15:14,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:15:14,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af9a6b275c8d3b7939b024 HTTP/1.1" 200 None 2026-03-10 04:15:14,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:15:14,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 04:15:14,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:15:14,302 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af9a6b275c8d3b7939b024 HTTP/1.1" 200 None 2026-03-10 04:15:14,381 - lava_callback - INFO - Completed processing callback for node 69af9a6b275c8d3b7939b024 INFO: 172.20.0.1:33938 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33946 - "GET / HTTP/1.0" 200 OK 2026-03-10 04:15:24,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:15:24,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:53370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35948 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36936 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57778 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37240 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54812 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54822 - "GET / HTTP/1.0" 200 OK 2026-03-10 04:20:22,013 - lava_callback - INFO - Received callback for node 69acf8f7bcd207e915d78752 INFO: 172.20.0.1:47240 - "POST /node/69acf8f7bcd207e915d78752 HTTP/1.0" 202 Accepted 2026-03-10 04:20:22,019 - lava_callback - INFO - Starting async job submit for node 69acf8f7bcd207e915d78752 2026-03-10 04:20:22,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:20:22,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acf8f7bcd207e915d78752 HTTP/1.1" 200 None 2026-03-10 04:20:22,036 - lava_callback - DEBUG - Processing node 69acf8f7bcd207e915d78752: baseline-arm-baylibre 2026-03-10 04:20:22,036 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acf8f7bcd207e915d78752 2026-03-10 04:20:22,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:20:22,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acf8f7bcd207e915d78752 HTTP/1.1" 200 None 2026-03-10 04:20:22,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:20:22,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a750a55a0c0e10c481a794 HTTP/1.1" 200 None 2026-03-10 04:20:22,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:20:22,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acf8f7bcd207e915d78752 HTTP/1.1" 200 None 2026-03-10 04:20:22,100 - lava_callback - ERROR - Error processing callback for node 69acf8f7bcd207e915d78752: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59538 - "GET / HTTP/1.0" 200 OK 2026-03-10 04:21:02,005 - lava_callback - INFO - Received callback for node 69acf920bcd207e915d78768 INFO: 172.20.0.1:55460 - "POST /node/69acf920bcd207e915d78768 HTTP/1.0" 202 Accepted 2026-03-10 04:21:02,011 - lava_callback - INFO - Starting async job submit for node 69acf920bcd207e915d78768 2026-03-10 04:21:02,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:21:02,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acf920bcd207e915d78768 HTTP/1.1" 200 None 2026-03-10 04:21:02,026 - lava_callback - DEBUG - Processing node 69acf920bcd207e915d78768: baseline-arm-baylibre 2026-03-10 04:21:02,027 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acf920bcd207e915d78768 2026-03-10 04:21:02,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:21:02,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acf920bcd207e915d78768 HTTP/1.1" 200 None 2026-03-10 04:21:02,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:21:02,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a750a55a0c0e10c481a794 HTTP/1.1" 200 None 2026-03-10 04:21:02,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:21:02,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acf920bcd207e915d78768 HTTP/1.1" 200 None 2026-03-10 04:21:02,078 - lava_callback - ERROR - Error processing callback for node 69acf920bcd207e915d78768: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54444 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52370 - "GET / HTTP/1.0" 200 OK 2026-03-10 04:21:42,086 - lava_callback - INFO - Received callback for node 69acf947bcd207e915d7877a INFO: 172.20.0.1:36244 - "POST /node/69acf947bcd207e915d7877a HTTP/1.0" 202 Accepted 2026-03-10 04:21:42,091 - lava_callback - INFO - Starting async job submit for node 69acf947bcd207e915d7877a 2026-03-10 04:21:42,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:21:42,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acf947bcd207e915d7877a HTTP/1.1" 200 None 2026-03-10 04:21:42,109 - lava_callback - DEBUG - Processing node 69acf947bcd207e915d7877a: baseline-arm-baylibre 2026-03-10 04:21:42,109 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acf947bcd207e915d7877a 2026-03-10 04:21:42,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:21:42,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acf947bcd207e915d7877a HTTP/1.1" 200 None 2026-03-10 04:21:42,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:21:42,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a750a55a0c0e10c481a794 HTTP/1.1" 200 None 2026-03-10 04:21:42,140 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:21:42,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acf947bcd207e915d7877a HTTP/1.1" 200 None 2026-03-10 04:21:42,193 - lava_callback - ERROR - Error processing callback for node 69acf947bcd207e915d7877a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45804 - "GET / HTTP/1.0" 200 OK 2026-03-10 04:22:00,847 - lava_callback - INFO - Received callback for node 69af9bf9275c8d3b7939b051 INFO: 172.20.0.1:57288 - "POST /node/69af9bf9275c8d3b7939b051 HTTP/1.0" 202 Accepted 2026-03-10 04:22:00,854 - lava_callback - INFO - Starting async job submit for node 69af9bf9275c8d3b7939b051 2026-03-10 04:22:00,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:22:00,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af9bf9275c8d3b7939b051 HTTP/1.1" 200 None 2026-03-10 04:22:00,903 - lava_callback - DEBUG - Processing node 69af9bf9275c8d3b7939b051: baseline-nfs-arm64 2026-03-10 04:22:01,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 04:22:01,456 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 04:22:01,458 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af9bf9275c8d3b7939b051/log.txt.gz 2026-03-10 04:22:01,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 04:22:01,955 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 04:22:01,957 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af9bf9275c8d3b7939b051/lava-logs.yaml 2026-03-10 04:22:01,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 04:22:02,123 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 04:22:02,125 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af9bf9275c8d3b7939b051/lava_callback.json.gz DEBUG: Setup failed for 69af9bf9275c8d3b7939b051. Transitting job node result: incomplete -> fail 2026-03-10 04:22:02,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:22:02,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af9bf9275c8d3b7939b051 HTTP/1.1" 200 None 2026-03-10 04:22:02,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:22:02,220 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 04:22:02,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:22:02,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af9bf9275c8d3b7939b051 HTTP/1.1" 200 None 2026-03-10 04:22:02,332 - lava_callback - INFO - Completed processing callback for node 69af9bf9275c8d3b7939b051 INFO: 172.20.0.1:57296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50850 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50852 - "GET / HTTP/1.0" 200 OK 2026-03-10 04:22:22,076 - lava_callback - INFO - Received callback for node 69acf970bcd207e915d78791 INFO: 172.20.0.1:44374 - "POST /node/69acf970bcd207e915d78791 HTTP/1.0" 202 Accepted 2026-03-10 04:22:22,082 - lava_callback - INFO - Starting async job submit for node 69acf970bcd207e915d78791 2026-03-10 04:22:22,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:22:22,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acf970bcd207e915d78791 HTTP/1.1" 200 None 2026-03-10 04:22:22,102 - lava_callback - DEBUG - Processing node 69acf970bcd207e915d78791: baseline-arm-baylibre 2026-03-10 04:22:22,102 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acf970bcd207e915d78791 2026-03-10 04:22:22,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:22:22,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acf970bcd207e915d78791 HTTP/1.1" 200 None 2026-03-10 04:22:22,119 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:22:22,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a750a55a0c0e10c481a794 HTTP/1.1" 200 None 2026-03-10 04:22:22,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:22:22,157 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acf970bcd207e915d78791 HTTP/1.1" 200 None 2026-03-10 04:22:22,163 - lava_callback - ERROR - Error processing callback for node 69acf970bcd207e915d78791: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-10 04:22:24,351 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:22:24,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:44382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38986 - "GET / HTTP/1.0" 200 OK 2026-03-10 04:23:02,232 - lava_callback - INFO - Received callback for node 69acf999bcd207e915d787a9 INFO: 172.20.0.1:54670 - "POST /node/69acf999bcd207e915d787a9 HTTP/1.0" 202 Accepted 2026-03-10 04:23:02,238 - lava_callback - INFO - Starting async job submit for node 69acf999bcd207e915d787a9 2026-03-10 04:23:02,239 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:23:02,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acf999bcd207e915d787a9 HTTP/1.1" 200 None 2026-03-10 04:23:02,254 - lava_callback - DEBUG - Processing node 69acf999bcd207e915d787a9: baseline-arm-baylibre 2026-03-10 04:23:02,255 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acf999bcd207e915d787a9 2026-03-10 04:23:02,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:23:02,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acf999bcd207e915d787a9 HTTP/1.1" 200 None 2026-03-10 04:23:02,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:23:02,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a750a55a0c0e10c481a794 HTTP/1.1" 200 None 2026-03-10 04:23:02,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:23:02,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acf999bcd207e915d787a9 HTTP/1.1" 200 None 2026-03-10 04:23:02,318 - lava_callback - ERROR - Error processing callback for node 69acf999bcd207e915d787a9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36668 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52354 - "GET / HTTP/1.0" 200 OK 2026-03-10 04:23:42,153 - lava_callback - INFO - Received callback for node 69acf9c1bcd207e915d787bf INFO: 172.20.0.1:35612 - "POST /node/69acf9c1bcd207e915d787bf HTTP/1.0" 202 Accepted 2026-03-10 04:23:42,158 - lava_callback - INFO - Starting async job submit for node 69acf9c1bcd207e915d787bf 2026-03-10 04:23:42,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:23:42,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acf9c1bcd207e915d787bf HTTP/1.1" 200 None 2026-03-10 04:23:42,174 - lava_callback - DEBUG - Processing node 69acf9c1bcd207e915d787bf: baseline-arm-baylibre 2026-03-10 04:23:42,175 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acf9c1bcd207e915d787bf 2026-03-10 04:23:42,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:23:42,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acf9c1bcd207e915d787bf HTTP/1.1" 200 None 2026-03-10 04:23:42,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:23:42,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a750a55a0c0e10c481a794 HTTP/1.1" 200 None 2026-03-10 04:23:42,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:23:42,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acf9c1bcd207e915d787bf HTTP/1.1" 200 None 2026-03-10 04:23:42,236 - lava_callback - ERROR - Error processing callback for node 69acf9c1bcd207e915d787bf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44610 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43074 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40248 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48610 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59762 - "GET / HTTP/1.0" 200 OK 2026-03-10 04:27:37,121 - lava_callback - INFO - Received callback for node 69af9dac275c8d3b7939b087 INFO: 172.20.0.1:59764 - "POST /node/69af9dac275c8d3b7939b087 HTTP/1.0" 202 Accepted 2026-03-10 04:27:37,128 - lava_callback - INFO - Starting async job submit for node 69af9dac275c8d3b7939b087 2026-03-10 04:27:37,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:27:37,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af9dac275c8d3b7939b087 HTTP/1.1" 200 None 2026-03-10 04:27:37,175 - lava_callback - DEBUG - Processing node 69af9dac275c8d3b7939b087: baseline-arm64-foundriesio 2026-03-10 04:27:37,224 - lava_callback - WARNING - Failed to upload log 2026-03-10 04:27:37,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 04:27:37,434 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 04:27:37,436 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69af9dac275c8d3b7939b087/lava-logs.yaml 2026-03-10 04:27:37,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 04:27:37,573 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 04:27:37,575 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69af9dac275c8d3b7939b087/lava_callback.json.gz 2026-03-10 04:27:37,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:27:37,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af9dac275c8d3b7939b087 HTTP/1.1" 200 None 2026-03-10 04:27:37,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:27:37,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 04:27:37,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:27:37,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af9dac275c8d3b7939b087 HTTP/1.1" 200 None 2026-03-10 04:27:37,715 - lava_callback - INFO - Completed processing callback for node 69af9dac275c8d3b7939b087 INFO: 172.20.0.1:38142 - "GET / HTTP/1.0" 200 OK 2026-03-10 04:27:54,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:27:54,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:37478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49494 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37556 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55450 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36748 - "GET / HTTP/1.0" 200 OK 2026-03-10 04:31:10,118 - lava_callback - INFO - Received callback for node 69af9e34275c8d3b7939b095 INFO: 172.20.0.1:45206 - "POST /node/69af9e34275c8d3b7939b095 HTTP/1.0" 202 Accepted 2026-03-10 04:31:10,125 - lava_callback - INFO - Starting async job submit for node 69af9e34275c8d3b7939b095 2026-03-10 04:31:10,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:31:10,187 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af9e34275c8d3b7939b095 HTTP/1.1" 200 None 2026-03-10 04:31:10,188 - lava_callback - DEBUG - Processing node 69af9e34275c8d3b7939b095: baseline-arm64 2026-03-10 04:31:11,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 04:31:11,168 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 04:31:11,170 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af9e34275c8d3b7939b095/log.txt.gz 2026-03-10 04:31:11,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 04:31:11,876 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 04:31:11,878 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af9e34275c8d3b7939b095/lava-logs.yaml 2026-03-10 04:31:11,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 04:31:12,054 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 04:31:12,056 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af9e34275c8d3b7939b095/lava_callback.json.gz DEBUG: Setup failed for 69af9e34275c8d3b7939b095. Transitting job node result: incomplete -> fail 2026-03-10 04:31:12,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:31:12,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af9e34275c8d3b7939b095 HTTP/1.1" 200 None 2026-03-10 04:31:12,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:31:12,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 04:31:12,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:31:12,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af9e34275c8d3b7939b095 HTTP/1.1" 200 None 2026-03-10 04:31:12,249 - lava_callback - INFO - Completed processing callback for node 69af9e34275c8d3b7939b095 INFO: 172.20.0.1:45210 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45224 - "GET / HTTP/1.0" 200 OK 2026-03-10 04:31:24,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:31:24,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:37638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45670 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51188 - "GET / HTTP/1.0" 200 OK 2026-03-10 04:32:42,499 - lava_callback - INFO - Received callback for node 69af9e86275c8d3b7939b09e INFO: 172.20.0.1:46402 - "POST /node/69af9e86275c8d3b7939b09e HTTP/1.0" 202 Accepted 2026-03-10 04:32:42,506 - lava_callback - INFO - Starting async job submit for node 69af9e86275c8d3b7939b09e 2026-03-10 04:32:42,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:32:42,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af9e86275c8d3b7939b09e HTTP/1.1" 200 None 2026-03-10 04:32:42,553 - lava_callback - DEBUG - Processing node 69af9e86275c8d3b7939b09e: baseline-nfs-arm64 2026-03-10 04:32:43,376 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 04:32:43,523 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 04:32:43,525 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af9e86275c8d3b7939b09e/log.txt.gz 2026-03-10 04:32:44,000 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 04:32:44,249 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 04:32:44,252 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af9e86275c8d3b7939b09e/lava-logs.yaml 2026-03-10 04:32:44,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 04:32:44,438 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 04:32:44,440 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af9e86275c8d3b7939b09e/lava_callback.json.gz DEBUG: Setup failed for 69af9e86275c8d3b7939b09e. Transitting job node result: incomplete -> fail 2026-03-10 04:32:44,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:32:44,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af9e86275c8d3b7939b09e HTTP/1.1" 200 None 2026-03-10 04:32:44,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:32:44,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 04:32:44,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:32:44,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af9e86275c8d3b7939b09e HTTP/1.1" 200 None 2026-03-10 04:32:44,660 - lava_callback - INFO - Completed processing callback for node 69af9e86275c8d3b7939b09e INFO: 172.20.0.1:46404 - "GET / HTTP/1.0" 200 OK 2026-03-10 04:32:54,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:32:54,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:49914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38486 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40334 - "GET / HTTP/1.0" 200 OK 2026-03-10 04:33:56,807 - lava_callback - INFO - Received callback for node 69af9df3275c8d3b7939b090 INFO: 172.20.0.1:40342 - "POST /node/69af9df3275c8d3b7939b090 HTTP/1.0" 202 Accepted 2026-03-10 04:33:56,813 - lava_callback - INFO - Starting async job submit for node 69af9df3275c8d3b7939b090 2026-03-10 04:33:56,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:33:56,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af9df3275c8d3b7939b090 HTTP/1.1" 200 None 2026-03-10 04:33:56,865 - lava_callback - DEBUG - Processing node 69af9df3275c8d3b7939b090: baseline-arm64-broonie 2026-03-10 04:33:57,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 04:33:57,485 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 04:33:57,486 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af9df3275c8d3b7939b090/log.txt.gz 2026-03-10 04:33:57,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 04:33:57,950 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 04:33:57,952 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af9df3275c8d3b7939b090/lava-logs.yaml 2026-03-10 04:33:57,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 04:33:58,136 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 04:33:58,137 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af9df3275c8d3b7939b090/lava_callback.json.gz DEBUG: Setup failed for 69af9df3275c8d3b7939b090. Transitting job node result: incomplete -> fail 2026-03-10 04:33:58,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:33:58,225 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af9df3275c8d3b7939b090 HTTP/1.1" 200 None 2026-03-10 04:33:58,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:33:58,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 04:33:58,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:33:58,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af9df3275c8d3b7939b090 HTTP/1.1" 200 None 2026-03-10 04:33:58,359 - lava_callback - INFO - Completed processing callback for node 69af9df3275c8d3b7939b090 INFO: 172.20.0.1:44236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55646 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55660 - "GET / HTTP/1.0" 200 OK 2026-03-10 04:34:24,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:34:24,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:45210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43528 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35100 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45950 - "GET / HTTP/1.0" 200 OK 2026-03-10 04:36:35,473 - lava_callback - INFO - Received callback for node 69af9ee0275c8d3b7939b0ae INFO: 172.20.0.1:56752 - "POST /node/69af9ee0275c8d3b7939b0ae HTTP/1.0" 202 Accepted 2026-03-10 04:36:35,479 - lava_callback - INFO - Starting async job submit for node 69af9ee0275c8d3b7939b0ae 2026-03-10 04:36:35,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:36:35,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af9ee0275c8d3b7939b0ae HTTP/1.1" 200 None 2026-03-10 04:36:35,529 - lava_callback - DEBUG - Processing node 69af9ee0275c8d3b7939b0ae: baseline-arm64-broonie 2026-03-10 04:36:35,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 04:36:36,110 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 04:36:36,111 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af9ee0275c8d3b7939b0ae/log.txt.gz 2026-03-10 04:36:36,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:56756 - "GET / HTTP/1.0" 200 OK 2026-03-10 04:36:36,686 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 04:36:36,688 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af9ee0275c8d3b7939b0ae/lava-logs.yaml 2026-03-10 04:36:36,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 04:36:36,841 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 04:36:36,842 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af9ee0275c8d3b7939b0ae/lava_callback.json.gz DEBUG: Setup failed for 69af9ee0275c8d3b7939b0ae. Transitting job node result: incomplete -> fail 2026-03-10 04:36:36,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:36:36,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af9ee0275c8d3b7939b0ae HTTP/1.1" 200 None 2026-03-10 04:36:36,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:36:36,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 04:36:36,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:36:36,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af9ee0275c8d3b7939b0ae HTTP/1.1" 200 None 2026-03-10 04:36:37,087 - lava_callback - INFO - Completed processing callback for node 69af9ee0275c8d3b7939b0ae INFO: 172.20.0.1:38114 - "GET / HTTP/1.0" 200 OK 2026-03-10 04:36:54,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:36:54,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:50458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41994 - "GET / HTTP/1.0" 200 OK 2026-03-10 04:37:08,605 - lava_callback - INFO - Received callback for node 69af9f78275c8d3b7939b0c7 INFO: 172.20.0.1:42006 - "POST /node/69af9f78275c8d3b7939b0c7 HTTP/1.0" 202 Accepted 2026-03-10 04:37:08,611 - lava_callback - INFO - Starting async job submit for node 69af9f78275c8d3b7939b0c7 2026-03-10 04:37:08,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:37:08,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af9f78275c8d3b7939b0c7 HTTP/1.1" 200 None 2026-03-10 04:37:08,665 - lava_callback - DEBUG - Processing node 69af9f78275c8d3b7939b0c7: baseline-arm64-broonie 2026-03-10 04:37:08,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 04:37:09,035 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 04:37:09,036 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af9f78275c8d3b7939b0c7/log.txt.gz 2026-03-10 04:37:09,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 04:37:13,878 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 04:37:13,880 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af9f78275c8d3b7939b0c7/lava-logs.yaml 2026-03-10 04:37:13,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 04:37:14,031 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 04:37:14,033 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af9f78275c8d3b7939b0c7/lava_callback.json.gz DEBUG: Setup failed for 69af9f78275c8d3b7939b0c7. Transitting job node result: incomplete -> fail 2026-03-10 04:37:14,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:37:14,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af9f78275c8d3b7939b0c7 HTTP/1.1" 200 None 2026-03-10 04:37:14,119 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:37:14,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 04:37:14,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:37:14,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af9f78275c8d3b7939b0c7 HTTP/1.1" 200 None 2026-03-10 04:37:14,231 - lava_callback - INFO - Completed processing callback for node 69af9f78275c8d3b7939b0c7 INFO: 172.20.0.1:53102 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53108 - "GET / HTTP/1.0" 200 OK 2026-03-10 04:37:24,461 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:37:24,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:40036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44234 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47130 - "GET / HTTP/1.0" 200 OK 2026-03-10 04:38:48,379 - lava_callback - INFO - Received callback for node 69af9f2b275c8d3b7939b0b8 INFO: 172.20.0.1:47142 - "POST /node/69af9f2b275c8d3b7939b0b8 HTTP/1.0" 202 Accepted 2026-03-10 04:38:48,385 - lava_callback - INFO - Starting async job submit for node 69af9f2b275c8d3b7939b0b8 2026-03-10 04:38:48,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:38:48,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af9f2b275c8d3b7939b0b8 HTTP/1.1" 200 None 2026-03-10 04:38:48,452 - lava_callback - DEBUG - Processing node 69af9f2b275c8d3b7939b0b8: baseline-arm64-broonie 2026-03-10 04:38:48,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 04:38:48,851 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 04:38:48,852 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af9f2b275c8d3b7939b0b8/log.txt.gz 2026-03-10 04:38:49,000 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 04:38:49,193 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 04:38:49,195 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af9f2b275c8d3b7939b0b8/lava-logs.yaml 2026-03-10 04:38:49,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 04:38:49,391 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 04:38:49,392 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af9f2b275c8d3b7939b0b8/lava_callback.json.gz DEBUG: Setup failed for 69af9f2b275c8d3b7939b0b8. Transitting job node result: incomplete -> fail 2026-03-10 04:38:49,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:38:49,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af9f2b275c8d3b7939b0b8 HTTP/1.1" 200 None 2026-03-10 04:38:49,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:38:49,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 04:38:49,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:38:49,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af9f2b275c8d3b7939b0b8 HTTP/1.1" 200 None 2026-03-10 04:38:49,632 - lava_callback - INFO - Completed processing callback for node 69af9f2b275c8d3b7939b0b8 2026-03-10 04:38:54,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:38:54,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:40118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33924 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33926 - "GET / HTTP/1.0" 200 OK 2026-03-10 04:39:23,101 - lava_callback - INFO - Received callback for node 69acfd75bcd207e915d78885 INFO: 172.20.0.1:59090 - "POST /node/69acfd75bcd207e915d78885 HTTP/1.0" 202 Accepted 2026-03-10 04:39:23,107 - lava_callback - INFO - Starting async job submit for node 69acfd75bcd207e915d78885 2026-03-10 04:39:23,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:39:23,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acfd75bcd207e915d78885 HTTP/1.1" 200 None 2026-03-10 04:39:23,124 - lava_callback - DEBUG - Processing node 69acfd75bcd207e915d78885: baseline-arm64-baylibre 2026-03-10 04:39:23,124 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acfd75bcd207e915d78885 2026-03-10 04:39:23,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:39:23,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acfd75bcd207e915d78885 HTTP/1.1" 200 None 2026-03-10 04:39:23,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:39:23,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a750ca5a0c0e10c481a7c4 HTTP/1.1" 200 None 2026-03-10 04:39:23,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:39:23,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acfd75bcd207e915d78885 HTTP/1.1" 200 None 2026-03-10 04:39:23,200 - lava_callback - ERROR - Error processing callback for node 69acfd75bcd207e915d78885: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52284 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60226 - "GET / HTTP/1.0" 200 OK 2026-03-10 04:40:43,155 - lava_callback - INFO - Received callback for node 69acfdc6bcd207e915d7889c INFO: 172.20.0.1:49404 - "POST /node/69acfdc6bcd207e915d7889c HTTP/1.0" 202 Accepted 2026-03-10 04:40:43,161 - lava_callback - INFO - Starting async job submit for node 69acfdc6bcd207e915d7889c 2026-03-10 04:40:43,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:40:43,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acfdc6bcd207e915d7889c HTTP/1.1" 200 None 2026-03-10 04:40:43,182 - lava_callback - DEBUG - Processing node 69acfdc6bcd207e915d7889c: baseline-arm64-baylibre 2026-03-10 04:40:43,182 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acfdc6bcd207e915d7889c 2026-03-10 04:40:43,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:40:43,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acfdc6bcd207e915d7889c HTTP/1.1" 200 None 2026-03-10 04:40:43,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:40:43,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a750ca5a0c0e10c481a7c4 HTTP/1.1" 200 None 2026-03-10 04:40:43,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:40:43,236 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acfdc6bcd207e915d7889c HTTP/1.1" 200 None 2026-03-10 04:40:43,239 - lava_callback - ERROR - Error processing callback for node 69acfdc6bcd207e915d7889c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60218 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44506 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49796 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49812 - "GET / HTTP/1.0" 200 OK 2026-03-10 04:43:23,285 - lava_callback - INFO - Received callback for node 69acfe65bcd207e915d788c3 INFO: 172.20.0.1:56296 - "POST /node/69acfe65bcd207e915d788c3 HTTP/1.0" 202 Accepted 2026-03-10 04:43:23,291 - lava_callback - INFO - Starting async job submit for node 69acfe65bcd207e915d788c3 2026-03-10 04:43:23,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:43:23,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acfe65bcd207e915d788c3 HTTP/1.1" 200 None 2026-03-10 04:43:23,312 - lava_callback - DEBUG - Processing node 69acfe65bcd207e915d788c3: baseline-arm64-baylibre 2026-03-10 04:43:23,312 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acfe65bcd207e915d788c3 2026-03-10 04:43:23,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:43:23,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acfe65bcd207e915d788c3 HTTP/1.1" 200 None 2026-03-10 04:43:23,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:43:23,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a750ca5a0c0e10c481a7c4 HTTP/1.1" 200 None 2026-03-10 04:43:23,351 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:43:23,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acfe65bcd207e915d788c3 HTTP/1.1" 200 None 2026-03-10 04:43:23,373 - lava_callback - ERROR - Error processing callback for node 69acfe65bcd207e915d788c3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52306 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60122 - "GET / HTTP/1.0" 200 OK 2026-03-10 04:44:49,081 - lava_callback - INFO - Received callback for node 69af9fc7275c8d3b7939b0cd INFO: 172.20.0.1:60132 - "POST /node/69af9fc7275c8d3b7939b0cd HTTP/1.0" 202 Accepted 2026-03-10 04:44:49,088 - lava_callback - INFO - Starting async job submit for node 69af9fc7275c8d3b7939b0cd 2026-03-10 04:44:49,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:44:49,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af9fc7275c8d3b7939b0cd HTTP/1.1" 200 None 2026-03-10 04:44:49,140 - lava_callback - DEBUG - Processing node 69af9fc7275c8d3b7939b0cd: baseline-arm64-broonie 2026-03-10 04:44:49,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 04:44:49,685 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 04:44:49,687 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af9fc7275c8d3b7939b0cd/log.txt.gz 2026-03-10 04:44:49,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 04:44:50,101 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 04:44:50,103 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af9fc7275c8d3b7939b0cd/lava-logs.yaml 2026-03-10 04:44:50,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 04:44:50,266 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 04:44:50,268 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af9fc7275c8d3b7939b0cd/lava_callback.json.gz DEBUG: Setup failed for 69af9fc7275c8d3b7939b0cd. Transitting job node result: incomplete -> fail 2026-03-10 04:44:50,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:44:50,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af9fc7275c8d3b7939b0cd HTTP/1.1" 200 None 2026-03-10 04:44:50,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:44:50,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 04:44:50,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:44:50,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af9fc7275c8d3b7939b0cd HTTP/1.1" 200 None 2026-03-10 04:44:50,481 - lava_callback - INFO - Completed processing callback for node 69af9fc7275c8d3b7939b0cd 2026-03-10 04:44:54,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:44:54,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:42952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33206 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33210 - "GET / HTTP/1.0" 200 OK 2026-03-10 04:45:23,442 - lava_callback - INFO - Received callback for node 69acfedfbcd207e915d788e0 INFO: 172.20.0.1:53010 - "POST /node/69acfedfbcd207e915d788e0 HTTP/1.0" 202 Accepted 2026-03-10 04:45:23,447 - lava_callback - INFO - Starting async job submit for node 69acfedfbcd207e915d788e0 2026-03-10 04:45:23,448 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:45:23,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acfedfbcd207e915d788e0 HTTP/1.1" 200 None 2026-03-10 04:45:23,489 - lava_callback - DEBUG - Processing node 69acfedfbcd207e915d788e0: baseline-arm64-baylibre 2026-03-10 04:45:23,489 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acfedfbcd207e915d788e0 2026-03-10 04:45:23,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:45:23,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acfedfbcd207e915d788e0 HTTP/1.1" 200 None 2026-03-10 04:45:23,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:45:23,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a750ca5a0c0e10c481a7c4 HTTP/1.1" 200 None 2026-03-10 04:45:23,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:45:23,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acfedfbcd207e915d788e0 HTTP/1.1" 200 None 2026-03-10 04:45:23,574 - lava_callback - ERROR - Error processing callback for node 69acfedfbcd207e915d788e0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35764 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42202 - "GET / HTTP/1.0" 200 OK 2026-03-10 04:46:43,785 - lava_callback - INFO - Received callback for node 69acff28bcd207e915d788f3 INFO: 172.20.0.1:36856 - "POST /node/69acff28bcd207e915d788f3 HTTP/1.0" 202 Accepted 2026-03-10 04:46:43,791 - lava_callback - INFO - Starting async job submit for node 69acff28bcd207e915d788f3 2026-03-10 04:46:43,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:46:43,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acff28bcd207e915d788f3 HTTP/1.1" 200 None 2026-03-10 04:46:43,811 - lava_callback - DEBUG - Processing node 69acff28bcd207e915d788f3: baseline-arm64-baylibre 2026-03-10 04:46:43,811 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acff28bcd207e915d788f3 2026-03-10 04:46:43,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:46:43,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acff28bcd207e915d788f3 HTTP/1.1" 200 None 2026-03-10 04:46:43,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:46:43,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a750ca5a0c0e10c481a7c4 HTTP/1.1" 200 None 2026-03-10 04:46:43,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:46:43,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acff28bcd207e915d788f3 HTTP/1.1" 200 None 2026-03-10 04:46:43,868 - lava_callback - ERROR - Error processing callback for node 69acff28bcd207e915d788f3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51376 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41498 - "GET / HTTP/1.0" 200 OK 2026-03-10 04:47:42,799 - lava_callback - INFO - Received callback for node 69afa009275c8d3b7939b0e2 INFO: 172.20.0.1:37832 - "POST /node/69afa009275c8d3b7939b0e2 HTTP/1.0" 202 Accepted 2026-03-10 04:47:42,806 - lava_callback - INFO - Starting async job submit for node 69afa009275c8d3b7939b0e2 2026-03-10 04:47:42,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:47:42,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afa009275c8d3b7939b0e2 HTTP/1.1" 200 None 2026-03-10 04:47:42,854 - lava_callback - DEBUG - Processing node 69afa009275c8d3b7939b0e2: baseline-nfs-arm64 2026-03-10 04:47:43,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 04:47:43,317 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 04:47:43,319 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69afa009275c8d3b7939b0e2/log.txt.gz 2026-03-10 04:47:43,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 04:47:43,736 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 04:47:43,738 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69afa009275c8d3b7939b0e2/lava-logs.yaml 2026-03-10 04:47:43,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 04:47:43,899 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 04:47:43,901 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69afa009275c8d3b7939b0e2/lava_callback.json.gz 2026-03-10 04:47:43,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:47:44,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afa009275c8d3b7939b0e2 HTTP/1.1" 200 None 2026-03-10 04:47:44,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:47:44,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 04:47:44,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:47:44,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afa009275c8d3b7939b0e2 HTTP/1.1" 200 None 2026-03-10 04:47:44,117 - lava_callback - INFO - Completed processing callback for node 69afa009275c8d3b7939b0e2 INFO: 172.20.0.1:37838 - "GET / HTTP/1.0" 200 OK 2026-03-10 04:47:54,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:47:54,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:55096 - "GET / HTTP/1.0" 200 OK 2026-03-10 04:48:03,556 - lava_callback - INFO - Received callback for node 69acff72bcd207e915d7890c INFO: 172.20.0.1:39192 - "POST /node/69acff72bcd207e915d7890c HTTP/1.0" 202 Accepted 2026-03-10 04:48:03,562 - lava_callback - INFO - Starting async job submit for node 69acff72bcd207e915d7890c 2026-03-10 04:48:03,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:48:03,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acff72bcd207e915d7890c HTTP/1.1" 200 None 2026-03-10 04:48:03,578 - lava_callback - DEBUG - Processing node 69acff72bcd207e915d7890c: baseline-arm64-baylibre 2026-03-10 04:48:03,578 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acff72bcd207e915d7890c 2026-03-10 04:48:03,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:48:03,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acff72bcd207e915d7890c HTTP/1.1" 200 None 2026-03-10 04:48:03,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:48:03,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a750ca5a0c0e10c481a7c4 HTTP/1.1" 200 None 2026-03-10 04:48:03,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:48:03,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acff72bcd207e915d7890c HTTP/1.1" 200 None 2026-03-10 04:48:03,652 - lava_callback - ERROR - Error processing callback for node 69acff72bcd207e915d7890c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48692 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46710 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33314 - "GET / HTTP/1.0" 200 OK 2026-03-10 04:49:38,110 - lava_callback - INFO - Received callback for node 69afa051275c8d3b7939b0e9 INFO: 172.20.0.1:33326 - "POST /node/69afa051275c8d3b7939b0e9 HTTP/1.0" 202 Accepted 2026-03-10 04:49:38,116 - lava_callback - INFO - Starting async job submit for node 69afa051275c8d3b7939b0e9 2026-03-10 04:49:38,147 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:49:38,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afa051275c8d3b7939b0e9 HTTP/1.1" 200 None 2026-03-10 04:49:38,162 - lava_callback - DEBUG - Processing node 69afa051275c8d3b7939b0e9: baseline-arm64 2026-03-10 04:49:38,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 04:49:38,626 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 04:49:38,628 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69afa051275c8d3b7939b0e9/log.txt.gz 2026-03-10 04:49:38,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 04:49:39,053 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 04:49:39,055 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69afa051275c8d3b7939b0e9/lava-logs.yaml 2026-03-10 04:49:39,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 04:49:39,223 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 04:49:39,225 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69afa051275c8d3b7939b0e9/lava_callback.json.gz 2026-03-10 04:49:39,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:49:39,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afa051275c8d3b7939b0e9 HTTP/1.1" 200 None 2026-03-10 04:49:39,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:49:39,286 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 04:49:39,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:49:39,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afa051275c8d3b7939b0e9 HTTP/1.1" 200 None 2026-03-10 04:49:39,380 - lava_callback - INFO - Completed processing callback for node 69afa051275c8d3b7939b0e9 INFO: 172.20.0.1:45810 - "GET / HTTP/1.0" 200 OK 2026-03-10 04:49:54,548 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:49:54,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:49902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50664 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42558 - "GET / HTTP/1.0" 200 OK 2026-03-10 04:50:43,715 - lava_callback - INFO - Received callback for node 69ad0010bcd207e915d78957 INFO: 172.20.0.1:54800 - "POST /node/69ad0010bcd207e915d78957 HTTP/1.0" 202 Accepted 2026-03-10 04:50:43,721 - lava_callback - INFO - Starting async job submit for node 69ad0010bcd207e915d78957 2026-03-10 04:50:43,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:50:43,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad0010bcd207e915d78957 HTTP/1.1" 200 None 2026-03-10 04:50:43,737 - lava_callback - DEBUG - Processing node 69ad0010bcd207e915d78957: baseline-arm64-baylibre 2026-03-10 04:50:43,737 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ad0010bcd207e915d78957 2026-03-10 04:50:43,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:50:43,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad0010bcd207e915d78957 HTTP/1.1" 200 None 2026-03-10 04:50:43,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:50:43,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a750ca5a0c0e10c481a7c4 HTTP/1.1" 200 None 2026-03-10 04:50:43,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:50:43,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ad0010bcd207e915d78957 HTTP/1.1" 200 None 2026-03-10 04:50:43,820 - lava_callback - ERROR - Error processing callback for node 69ad0010bcd207e915d78957: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44762 - "GET / HTTP/1.0" 200 OK 2026-03-10 04:51:11,532 - lava_callback - INFO - Received callback for node 69afa0a8275c8d3b7939b0f9 INFO: 172.20.0.1:51794 - "POST /node/69afa0a8275c8d3b7939b0f9 HTTP/1.0" 202 Accepted 2026-03-10 04:51:11,539 - lava_callback - INFO - Starting async job submit for node 69afa0a8275c8d3b7939b0f9 2026-03-10 04:51:11,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:51:11,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afa0a8275c8d3b7939b0f9 HTTP/1.1" 200 None 2026-03-10 04:51:11,579 - lava_callback - DEBUG - Processing node 69afa0a8275c8d3b7939b0f9: baseline-nfs-arm64 2026-03-10 04:51:12,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 04:51:12,282 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 04:51:12,283 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69afa0a8275c8d3b7939b0f9/log.txt.gz 2026-03-10 04:51:12,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 04:51:12,786 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 04:51:12,788 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69afa0a8275c8d3b7939b0f9/lava-logs.yaml 2026-03-10 04:51:12,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 04:51:13,000 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 04:51:13,002 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69afa0a8275c8d3b7939b0f9/lava_callback.json.gz 2026-03-10 04:51:13,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:51:13,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afa0a8275c8d3b7939b0f9 HTTP/1.1" 200 None 2026-03-10 04:51:13,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:51:13,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 04:51:13,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:51:13,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afa0a8275c8d3b7939b0f9 HTTP/1.1" 200 None 2026-03-10 04:51:13,142 - lava_callback - INFO - Completed processing callback for node 69afa0a8275c8d3b7939b0f9 INFO: 172.20.0.1:51798 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51808 - "GET / HTTP/1.0" 200 OK 2026-03-10 04:51:22,531 - lava_callback - INFO - Received callback for node 69afa0f1275c8d3b7939b103 INFO: 172.20.0.1:47328 - "POST /node/69afa0f1275c8d3b7939b103 HTTP/1.0" 202 Accepted 2026-03-10 04:51:22,540 - lava_callback - INFO - Starting async job submit for node 69afa0f1275c8d3b7939b103 2026-03-10 04:51:22,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:51:22,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afa0f1275c8d3b7939b103 HTTP/1.1" 200 None 2026-03-10 04:51:22,582 - lava_callback - DEBUG - Processing node 69afa0f1275c8d3b7939b103: baseline-arm64 2026-03-10 04:51:23,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 04:51:23,237 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 04:51:23,238 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69afa0f1275c8d3b7939b103/log.txt.gz 2026-03-10 04:51:23,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 04:51:23,690 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 04:51:23,693 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69afa0f1275c8d3b7939b103/lava-logs.yaml 2026-03-10 04:51:23,704 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 04:51:23,853 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 04:51:23,855 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69afa0f1275c8d3b7939b103/lava_callback.json.gz 2026-03-10 04:51:23,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:51:23,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afa0f1275c8d3b7939b103 HTTP/1.1" 200 None 2026-03-10 04:51:23,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:51:23,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 04:51:23,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:51:23,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afa0f1275c8d3b7939b103 HTTP/1.1" 200 None 2026-03-10 04:51:24,011 - lava_callback - INFO - Completed processing callback for node 69afa0f1275c8d3b7939b103 2026-03-10 04:51:24,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:51:24,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:47332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41630 - "GET / HTTP/1.0" 200 OK 2026-03-10 04:51:43,674 - lava_callback - INFO - Received callback for node 69ad005bbcd207e915d78974 INFO: 172.20.0.1:54678 - "POST /node/69ad005bbcd207e915d78974 HTTP/1.0" 202 Accepted 2026-03-10 04:51:43,680 - lava_callback - INFO - Starting async job submit for node 69ad005bbcd207e915d78974 2026-03-10 04:51:43,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:51:43,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad005bbcd207e915d78974 HTTP/1.1" 200 None 2026-03-10 04:51:43,699 - lava_callback - DEBUG - Processing node 69ad005bbcd207e915d78974: baseline-arm64-baylibre 2026-03-10 04:51:43,700 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ad005bbcd207e915d78974 2026-03-10 04:51:43,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:51:43,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad005bbcd207e915d78974 HTTP/1.1" 200 None 2026-03-10 04:51:43,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:51:43,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a750ca5a0c0e10c481a7c4 HTTP/1.1" 200 None 2026-03-10 04:51:43,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:51:43,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ad005bbcd207e915d78974 HTTP/1.1" 200 None 2026-03-10 04:51:43,764 - lava_callback - ERROR - Error processing callback for node 69ad005bbcd207e915d78974: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46956 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51686 - "GET / HTTP/1.0" 200 OK 2026-03-10 04:53:03,752 - lava_callback - INFO - Received callback for node 69ad00aebcd207e915d78989 INFO: 172.20.0.1:49046 - "POST /node/69ad00aebcd207e915d78989 HTTP/1.0" 202 Accepted 2026-03-10 04:53:03,758 - lava_callback - INFO - Starting async job submit for node 69ad00aebcd207e915d78989 2026-03-10 04:53:03,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:53:03,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad00aebcd207e915d78989 HTTP/1.1" 200 None 2026-03-10 04:53:03,778 - lava_callback - DEBUG - Processing node 69ad00aebcd207e915d78989: baseline-arm64-baylibre 2026-03-10 04:53:03,778 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ad00aebcd207e915d78989 2026-03-10 04:53:03,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:53:03,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad00aebcd207e915d78989 HTTP/1.1" 200 None 2026-03-10 04:53:03,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:53:03,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a750ca5a0c0e10c481a7c4 HTTP/1.1" 200 None 2026-03-10 04:53:03,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:53:03,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ad00aebcd207e915d78989 HTTP/1.1" 200 None 2026-03-10 04:53:03,835 - lava_callback - ERROR - Error processing callback for node 69ad00aebcd207e915d78989: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59784 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41400 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45328 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36994 - "GET / HTTP/1.0" 200 OK 2026-03-10 04:55:43,921 - lava_callback - INFO - Received callback for node 69ad0148bcd207e915d789de INFO: 172.20.0.1:34278 - "POST /node/69ad0148bcd207e915d789de HTTP/1.0" 202 Accepted 2026-03-10 04:55:43,926 - lava_callback - INFO - Starting async job submit for node 69ad0148bcd207e915d789de 2026-03-10 04:55:43,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:55:43,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad0148bcd207e915d789de HTTP/1.1" 200 None 2026-03-10 04:55:43,942 - lava_callback - DEBUG - Processing node 69ad0148bcd207e915d789de: baseline-arm64-baylibre 2026-03-10 04:55:43,942 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ad0148bcd207e915d789de 2026-03-10 04:55:43,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:55:43,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad0148bcd207e915d789de HTTP/1.1" 200 None 2026-03-10 04:55:43,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:55:43,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a750ca5a0c0e10c481a7c4 HTTP/1.1" 200 None 2026-03-10 04:55:43,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:55:44,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ad0148bcd207e915d789de HTTP/1.1" 200 None 2026-03-10 04:55:44,012 - lava_callback - ERROR - Error processing callback for node 69ad0148bcd207e915d789de: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49484 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42958 - "GET / HTTP/1.0" 200 OK 2026-03-10 04:56:49,816 - lava_callback - INFO - Received callback for node 69ad018dbcd207e915d789f8 INFO: 172.20.0.1:47054 - "POST /node/69ad018dbcd207e915d789f8 HTTP/1.0" 202 Accepted 2026-03-10 04:56:49,822 - lava_callback - INFO - Starting async job submit for node 69ad018dbcd207e915d789f8 2026-03-10 04:56:49,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:56:49,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad018dbcd207e915d789f8 HTTP/1.1" 200 None 2026-03-10 04:56:49,838 - lava_callback - DEBUG - Processing node 69ad018dbcd207e915d789f8: baseline-arm-clabbe 2026-03-10 04:56:49,838 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ad018dbcd207e915d789f8 2026-03-10 04:56:49,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:56:49,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad018dbcd207e915d789f8 HTTP/1.1" 200 None 2026-03-10 04:56:49,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:56:49,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac4f6846f7ae8ff3364c2c HTTP/1.1" 200 None 2026-03-10 04:56:49,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:56:49,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ad018dbcd207e915d789f8 HTTP/1.1" 200 None 2026-03-10 04:56:49,896 - lava_callback - ERROR - Error processing callback for node 69ad018dbcd207e915d789f8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35764 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35626 - "GET / HTTP/1.0" 200 OK 2026-03-10 04:57:29,761 - lava_callback - INFO - Received callback for node 69ad01aabcd207e915d78a11 INFO: 172.20.0.1:37754 - "POST /node/69ad01aabcd207e915d78a11 HTTP/1.0" 202 Accepted 2026-03-10 04:57:29,767 - lava_callback - INFO - Starting async job submit for node 69ad01aabcd207e915d78a11 2026-03-10 04:57:29,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:57:29,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad01aabcd207e915d78a11 HTTP/1.1" 200 None 2026-03-10 04:57:29,784 - lava_callback - DEBUG - Processing node 69ad01aabcd207e915d78a11: baseline-arm-clabbe 2026-03-10 04:57:29,784 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ad01aabcd207e915d78a11 2026-03-10 04:57:29,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:57:29,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad01aabcd207e915d78a11 HTTP/1.1" 200 None 2026-03-10 04:57:29,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:57:29,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac4f6846f7ae8ff3364c2c HTTP/1.1" 200 None 2026-03-10 04:57:29,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:57:29,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ad01aabcd207e915d78a11 HTTP/1.1" 200 None 2026-03-10 04:57:29,847 - lava_callback - ERROR - Error processing callback for node 69ad01aabcd207e915d78a11: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56590 - "GET / HTTP/1.0" 200 OK 2026-03-10 04:57:49,839 - lava_callback - INFO - Received callback for node 69ad01c7bcd207e915d78a25 INFO: 172.20.0.1:48898 - "POST /node/69ad01c7bcd207e915d78a25 HTTP/1.0" 202 Accepted 2026-03-10 04:57:49,845 - lava_callback - INFO - Starting async job submit for node 69ad01c7bcd207e915d78a25 2026-03-10 04:57:49,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:57:49,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad01c7bcd207e915d78a25 HTTP/1.1" 200 None 2026-03-10 04:57:49,861 - lava_callback - DEBUG - Processing node 69ad01c7bcd207e915d78a25: baseline-arm-clabbe 2026-03-10 04:57:49,861 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ad01c7bcd207e915d78a25 2026-03-10 04:57:49,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:57:49,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad01c7bcd207e915d78a25 HTTP/1.1" 200 None 2026-03-10 04:57:49,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:57:49,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac4f6846f7ae8ff3364c2c HTTP/1.1" 200 None 2026-03-10 04:57:49,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:57:49,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ad01c7bcd207e915d78a25 HTTP/1.1" 200 None 2026-03-10 04:57:49,933 - lava_callback - ERROR - Error processing callback for node 69ad01c7bcd207e915d78a25: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60734 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60750 - "GET / HTTP/1.0" 200 OK 2026-03-10 04:58:24,128 - lava_callback - INFO - Received callback for node 69ad01ebbcd207e915d78a43 INFO: 172.20.0.1:38368 - "POST /node/69ad01ebbcd207e915d78a43 HTTP/1.0" 202 Accepted 2026-03-10 04:58:24,133 - lava_callback - INFO - Starting async job submit for node 69ad01ebbcd207e915d78a43 2026-03-10 04:58:24,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:58:24,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad01ebbcd207e915d78a43 HTTP/1.1" 200 None 2026-03-10 04:58:24,154 - lava_callback - DEBUG - Processing node 69ad01ebbcd207e915d78a43: baseline-arm64-baylibre 2026-03-10 04:58:24,154 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ad01ebbcd207e915d78a43 2026-03-10 04:58:24,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:58:24,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad01ebbcd207e915d78a43 HTTP/1.1" 200 None 2026-03-10 04:58:24,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:58:24,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a750ca5a0c0e10c481a7c4 HTTP/1.1" 200 None 2026-03-10 04:58:24,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:58:24,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ad01ebbcd207e915d78a43 HTTP/1.1" 200 None 2026-03-10 04:58:24,210 - lava_callback - ERROR - Error processing callback for node 69ad01ebbcd207e915d78a43: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58248 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41542 - "GET / HTTP/1.0" 200 OK 2026-03-10 04:59:44,159 - lava_callback - INFO - Received callback for node 69ad0235bcd207e915d78a5c INFO: 172.20.0.1:56078 - "POST /node/69ad0235bcd207e915d78a5c HTTP/1.0" 202 Accepted 2026-03-10 04:59:44,164 - lava_callback - INFO - Starting async job submit for node 69ad0235bcd207e915d78a5c 2026-03-10 04:59:44,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:59:44,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad0235bcd207e915d78a5c HTTP/1.1" 200 None 2026-03-10 04:59:44,180 - lava_callback - DEBUG - Processing node 69ad0235bcd207e915d78a5c: baseline-arm64-baylibre 2026-03-10 04:59:44,180 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ad0235bcd207e915d78a5c 2026-03-10 04:59:44,181 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:59:44,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad0235bcd207e915d78a5c HTTP/1.1" 200 None 2026-03-10 04:59:44,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:59:44,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a750ca5a0c0e10c481a7c4 HTTP/1.1" 200 None 2026-03-10 04:59:44,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 04:59:44,235 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ad0235bcd207e915d78a5c HTTP/1.1" 200 None 2026-03-10 04:59:44,237 - lava_callback - ERROR - Error processing callback for node 69ad0235bcd207e915d78a5c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50782 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45768 - "GET / HTTP/1.0" 200 OK 2026-03-10 05:00:49,989 - lava_callback - INFO - Received callback for node 69ad027abcd207e915d78a7b INFO: 172.20.0.1:52120 - "POST /node/69ad027abcd207e915d78a7b HTTP/1.0" 202 Accepted 2026-03-10 05:00:49,994 - lava_callback - INFO - Starting async job submit for node 69ad027abcd207e915d78a7b 2026-03-10 05:00:49,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:00:50,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad027abcd207e915d78a7b HTTP/1.1" 200 None 2026-03-10 05:00:50,010 - lava_callback - DEBUG - Processing node 69ad027abcd207e915d78a7b: baseline-arm-clabbe 2026-03-10 05:00:50,011 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ad027abcd207e915d78a7b 2026-03-10 05:00:50,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:00:50,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad027abcd207e915d78a7b HTTP/1.1" 200 None 2026-03-10 05:00:50,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:00:50,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a9acd99c5f93e4f05eb2ac HTTP/1.1" 200 None 2026-03-10 05:00:50,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:00:50,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ad027abcd207e915d78a7b HTTP/1.1" 200 None 2026-03-10 05:00:50,064 - lava_callback - ERROR - Error processing callback for node 69ad027abcd207e915d78a7b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38444 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38452 - "GET / HTTP/1.0" 200 OK 2026-03-10 05:01:24,586 - lava_callback - INFO - Received callback for node 69ad029fbcd207e915d78aa2 INFO: 172.20.0.1:34660 - "POST /node/69ad029fbcd207e915d78aa2 HTTP/1.0" 202 Accepted 2026-03-10 05:01:24,591 - lava_callback - INFO - Starting async job submit for node 69ad029fbcd207e915d78aa2 2026-03-10 05:01:24,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:01:24,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad029fbcd207e915d78aa2 HTTP/1.1" 200 None 2026-03-10 05:01:24,608 - lava_callback - DEBUG - Processing node 69ad029fbcd207e915d78aa2: baseline-arm64-baylibre 2026-03-10 05:01:24,608 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ad029fbcd207e915d78aa2 2026-03-10 05:01:24,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:01:24,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad029fbcd207e915d78aa2 HTTP/1.1" 200 None 2026-03-10 05:01:24,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:01:24,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a750ca5a0c0e10c481a7c4 HTTP/1.1" 200 None 2026-03-10 05:01:24,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:01:24,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ad029fbcd207e915d78aa2 HTTP/1.1" 200 None 2026-03-10 05:01:24,683 - lava_callback - ERROR - Error processing callback for node 69ad029fbcd207e915d78aa2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58536 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39550 - "GET / HTTP/1.0" 200 OK 2026-03-10 05:02:30,049 - lava_callback - INFO - Received callback for node 69ad02e3bcd207e915d78ab0 INFO: 172.20.0.1:40420 - "POST /node/69ad02e3bcd207e915d78ab0 HTTP/1.0" 202 Accepted 2026-03-10 05:02:30,054 - lava_callback - INFO - Starting async job submit for node 69ad02e3bcd207e915d78ab0 2026-03-10 05:02:30,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:02:30,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad02e3bcd207e915d78ab0 HTTP/1.1" 200 None 2026-03-10 05:02:30,094 - lava_callback - DEBUG - Processing node 69ad02e3bcd207e915d78ab0: baseline-arm-clabbe 2026-03-10 05:02:30,094 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ad02e3bcd207e915d78ab0 2026-03-10 05:02:30,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:02:30,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad02e3bcd207e915d78ab0 HTTP/1.1" 200 None 2026-03-10 05:02:30,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:02:30,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a9acd99c5f93e4f05eb2ac HTTP/1.1" 200 None 2026-03-10 05:02:30,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:02:30,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ad02e3bcd207e915d78ab0 HTTP/1.1" 200 None 2026-03-10 05:02:30,160 - lava_callback - ERROR - Error processing callback for node 69ad02e3bcd207e915d78ab0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51986 - "GET / HTTP/1.0" 200 OK 2026-03-10 05:03:10,075 - lava_callback - INFO - Received callback for node 69ad0300bcd207e915d78ac2 INFO: 172.20.0.1:57024 - "POST /node/69ad0300bcd207e915d78ac2 HTTP/1.0" 202 Accepted 2026-03-10 05:03:10,081 - lava_callback - INFO - Starting async job submit for node 69ad0300bcd207e915d78ac2 2026-03-10 05:03:10,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:03:10,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad0300bcd207e915d78ac2 HTTP/1.1" 200 None 2026-03-10 05:03:10,099 - lava_callback - DEBUG - Processing node 69ad0300bcd207e915d78ac2: baseline-arm-clabbe 2026-03-10 05:03:10,099 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ad0300bcd207e915d78ac2 2026-03-10 05:03:10,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:03:10,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad0300bcd207e915d78ac2 HTTP/1.1" 200 None 2026-03-10 05:03:10,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:03:10,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a9acd99c5f93e4f05eb2ac HTTP/1.1" 200 None 2026-03-10 05:03:10,130 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:03:10,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ad0300bcd207e915d78ac2 HTTP/1.1" 200 None 2026-03-10 05:03:10,157 - lava_callback - ERROR - Error processing callback for node 69ad0300bcd207e915d78ac2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57028 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43936 - "GET / HTTP/1.0" 200 OK 2026-03-10 05:03:29,509 - lava_callback - INFO - Received callback for node 69af95bb275c8d3b7939af49 INFO: 172.20.0.1:43940 - "POST /node/69af95bb275c8d3b7939af49 HTTP/1.0" 202 Accepted 2026-03-10 05:03:29,515 - lava_callback - INFO - Starting async job submit for node 69af95bb275c8d3b7939af49 2026-03-10 05:03:29,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:03:29,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af95bb275c8d3b7939af49 HTTP/1.1" 200 None 2026-03-10 05:03:29,548 - lava_callback - DEBUG - Processing node 69af95bb275c8d3b7939af49: baseline-arm64-broonie 2026-03-10 05:03:29,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 05:03:30,048 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 05:03:30,050 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af95bb275c8d3b7939af49/log.txt.gz 2026-03-10 05:03:30,101 - lava_callback - INFO - Received callback for node 69ad031dbcd207e915d78adf INFO: 172.20.0.1:41382 - "POST /node/69ad031dbcd207e915d78adf HTTP/1.0" 202 Accepted 2026-03-10 05:03:30,118 - lava_callback - INFO - Starting async job submit for node 69ad031dbcd207e915d78adf 2026-03-10 05:03:30,130 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:03:30,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad031dbcd207e915d78adf HTTP/1.1" 200 None 2026-03-10 05:03:30,216 - lava_callback - DEBUG - Processing node 69ad031dbcd207e915d78adf: baseline-arm-clabbe 2026-03-10 05:03:30,222 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ad031dbcd207e915d78adf 2026-03-10 05:03:30,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:03:30,270 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 05:03:30,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad031dbcd207e915d78adf HTTP/1.1" 200 None 2026-03-10 05:03:30,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:03:30,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a9acd99c5f93e4f05eb2ac HTTP/1.1" 200 None 2026-03-10 05:03:30,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:03:30,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ad031dbcd207e915d78adf HTTP/1.1" 200 None 2026-03-10 05:03:30,336 - lava_callback - ERROR - Error processing callback for node 69ad031dbcd207e915d78adf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-10 05:03:30,534 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 05:03:30,536 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af95bb275c8d3b7939af49/lava-logs.yaml 2026-03-10 05:03:30,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 05:03:30,694 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 05:03:30,696 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af95bb275c8d3b7939af49/lava_callback.json.gz 2026-03-10 05:03:30,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:03:30,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af95bb275c8d3b7939af49 HTTP/1.1" 200 None 2026-03-10 05:03:30,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:03:30,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 05:03:30,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:03:30,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af95bb275c8d3b7939af49 HTTP/1.1" 200 None 2026-03-10 05:03:30,839 - lava_callback - INFO - Completed processing callback for node 69af95bb275c8d3b7939af49 INFO: 172.20.0.1:41396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57060 - "GET / HTTP/1.0" 200 OK 2026-03-10 05:03:54,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:03:54,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:36000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60010 - "GET / HTTP/1.0" 200 OK 2026-03-10 05:04:10,112 - lava_callback - INFO - Received callback for node 69ad0341bcd207e915d78af4 INFO: 172.20.0.1:36630 - "POST /node/69ad0341bcd207e915d78af4 HTTP/1.0" 202 Accepted 2026-03-10 05:04:10,117 - lava_callback - INFO - Starting async job submit for node 69ad0341bcd207e915d78af4 2026-03-10 05:04:10,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:04:10,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad0341bcd207e915d78af4 HTTP/1.1" 200 None 2026-03-10 05:04:10,134 - lava_callback - DEBUG - Processing node 69ad0341bcd207e915d78af4: baseline-arm-clabbe 2026-03-10 05:04:10,134 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ad0341bcd207e915d78af4 2026-03-10 05:04:10,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:04:10,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad0341bcd207e915d78af4 HTTP/1.1" 200 None 2026-03-10 05:04:10,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:04:10,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a9acd99c5f93e4f05eb2ac HTTP/1.1" 200 None 2026-03-10 05:04:10,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:04:10,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ad0341bcd207e915d78af4 HTTP/1.1" 200 None 2026-03-10 05:04:10,185 - lava_callback - ERROR - Error processing callback for node 69ad0341bcd207e915d78af4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36642 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49184 - "GET / HTTP/1.0" 200 OK 2026-03-10 05:04:50,161 - lava_callback - INFO - Received callback for node 69ad0368bcd207e915d78b0e INFO: 172.20.0.1:46194 - "POST /node/69ad0368bcd207e915d78b0e HTTP/1.0" 202 Accepted 2026-03-10 05:04:50,166 - lava_callback - INFO - Starting async job submit for node 69ad0368bcd207e915d78b0e 2026-03-10 05:04:50,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:04:50,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad0368bcd207e915d78b0e HTTP/1.1" 200 None 2026-03-10 05:04:50,189 - lava_callback - DEBUG - Processing node 69ad0368bcd207e915d78b0e: baseline-arm-clabbe 2026-03-10 05:04:50,189 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ad0368bcd207e915d78b0e 2026-03-10 05:04:50,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:04:50,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad0368bcd207e915d78b0e HTTP/1.1" 200 None 2026-03-10 05:04:50,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:04:50,225 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a9acd99c5f93e4f05eb2ac HTTP/1.1" 200 None 2026-03-10 05:04:50,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:04:50,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ad0368bcd207e915d78b0e HTTP/1.1" 200 None 2026-03-10 05:04:50,248 - lava_callback - ERROR - Error processing callback for node 69ad0368bcd207e915d78b0e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51980 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41306 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36734 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53370 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46230 - "GET / HTTP/1.0" 200 OK 2026-03-10 05:08:29,619 - lava_callback - INFO - Received callback for node 69afa688275c8d3b7939b29e INFO: 172.20.0.1:46234 - "POST /node/69afa688275c8d3b7939b29e HTTP/1.0" 202 Accepted 2026-03-10 05:08:29,625 - lava_callback - INFO - Starting async job submit for node 69afa688275c8d3b7939b29e 2026-03-10 05:08:29,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:08:29,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afa688275c8d3b7939b29e HTTP/1.1" 200 None 2026-03-10 05:08:29,668 - lava_callback - DEBUG - Processing node 69afa688275c8d3b7939b29e: baseline-arm64-broonie 2026-03-10 05:08:29,971 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 05:08:30,158 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 05:08:30,160 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69afa688275c8d3b7939b29e/log.txt.gz 2026-03-10 05:08:30,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 05:08:30,565 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 05:08:30,567 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69afa688275c8d3b7939b29e/lava-logs.yaml 2026-03-10 05:08:30,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 05:08:30,718 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 05:08:30,720 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69afa688275c8d3b7939b29e/lava_callback.json.gz 2026-03-10 05:08:30,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:08:30,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afa688275c8d3b7939b29e HTTP/1.1" 200 None 2026-03-10 05:08:30,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:08:30,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 05:08:30,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:08:30,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afa688275c8d3b7939b29e HTTP/1.1" 200 None 2026-03-10 05:08:30,849 - lava_callback - INFO - Completed processing callback for node 69afa688275c8d3b7939b29e INFO: 172.20.0.1:54992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52116 - "GET / HTTP/1.0" 200 OK 2026-03-10 05:08:54,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:08:54,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:49484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54480 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37720 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45168 - "GET / HTTP/1.0" 200 OK 2026-03-10 05:10:30,393 - lava_callback - INFO - Received callback for node 69ad04bebcd207e915d78b80 INFO: 172.20.0.1:56294 - "POST /node/69ad04bebcd207e915d78b80 HTTP/1.0" 202 Accepted 2026-03-10 05:10:30,398 - lava_callback - INFO - Starting async job submit for node 69ad04bebcd207e915d78b80 2026-03-10 05:10:30,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:10:30,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad04bebcd207e915d78b80 HTTP/1.1" 200 None 2026-03-10 05:10:30,428 - lava_callback - DEBUG - Processing node 69ad04bebcd207e915d78b80: baseline-arm-clabbe 2026-03-10 05:10:30,428 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ad04bebcd207e915d78b80 2026-03-10 05:10:30,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:10:30,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad04bebcd207e915d78b80 HTTP/1.1" 200 None 2026-03-10 05:10:30,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:10:30,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a9acd99c5f93e4f05eb2ac HTTP/1.1" 200 None 2026-03-10 05:10:30,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:10:30,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ad04bebcd207e915d78b80 HTTP/1.1" 200 None 2026-03-10 05:10:30,487 - lava_callback - ERROR - Error processing callback for node 69ad04bebcd207e915d78b80: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41270 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51342 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56396 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32910 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54916 - "GET / HTTP/1.0" 200 OK 2026-03-10 05:14:45,031 - lava_callback - INFO - Received callback for node 69ad05afbcd207e915d78bcb INFO: 172.20.0.1:46242 - "POST /node/69ad05afbcd207e915d78bcb HTTP/1.0" 202 Accepted 2026-03-10 05:14:45,036 - lava_callback - INFO - Starting async job submit for node 69ad05afbcd207e915d78bcb 2026-03-10 05:14:45,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:14:45,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad05afbcd207e915d78bcb HTTP/1.1" 200 None 2026-03-10 05:14:45,054 - lava_callback - DEBUG - Processing node 69ad05afbcd207e915d78bcb: baseline-arm64-baylibre 2026-03-10 05:14:45,054 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ad05afbcd207e915d78bcb 2026-03-10 05:14:45,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:14:45,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad05afbcd207e915d78bcb HTTP/1.1" 200 None 2026-03-10 05:14:45,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:14:45,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac4f7446f7ae8ff3364c45 HTTP/1.1" 200 None 2026-03-10 05:14:45,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:14:45,187 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ad05afbcd207e915d78bcb HTTP/1.1" 200 None 2026-03-10 05:14:45,188 - lava_callback - ERROR - Error processing callback for node 69ad05afbcd207e915d78bcb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39228 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58056 - "GET / HTTP/1.0" 200 OK 2026-03-10 05:15:45,043 - lava_callback - INFO - Received callback for node 69ad05fdbcd207e915d78bd8 INFO: 172.20.0.1:50370 - "POST /node/69ad05fdbcd207e915d78bd8 HTTP/1.0" 202 Accepted 2026-03-10 05:15:45,049 - lava_callback - INFO - Starting async job submit for node 69ad05fdbcd207e915d78bd8 2026-03-10 05:15:45,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:15:45,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad05fdbcd207e915d78bd8 HTTP/1.1" 200 None 2026-03-10 05:15:45,072 - lava_callback - DEBUG - Processing node 69ad05fdbcd207e915d78bd8: baseline-arm64-baylibre 2026-03-10 05:15:45,072 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ad05fdbcd207e915d78bd8 2026-03-10 05:15:45,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:15:45,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad05fdbcd207e915d78bd8 HTTP/1.1" 200 None 2026-03-10 05:15:45,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:15:45,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a9acf29c5f93e4f05eb2cc HTTP/1.1" 200 None 2026-03-10 05:15:45,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:15:45,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ad05fdbcd207e915d78bd8 HTTP/1.1" 200 None 2026-03-10 05:15:45,124 - lava_callback - ERROR - Error processing callback for node 69ad05fdbcd207e915d78bd8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32808 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36438 - "GET / HTTP/1.0" 200 OK 2026-03-10 05:16:30,431 - lava_callback - INFO - Received callback for node 69af67af275c8d3b79399ea9 INFO: 172.20.0.1:46020 - "POST /node/69af67af275c8d3b79399ea9 HTTP/1.0" 202 Accepted 2026-03-10 05:16:30,482 - lava_callback - INFO - Starting async job submit for node 69af67af275c8d3b79399ea9 2026-03-10 05:16:30,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:16:30,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af67af275c8d3b79399ea9 HTTP/1.1" 200 None 2026-03-10 05:16:30,527 - lava_callback - DEBUG - Processing node 69af67af275c8d3b79399ea9: baseline-arm-pengutronix 2026-03-10 05:16:30,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 05:16:30,794 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 05:16:30,796 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69af67af275c8d3b79399ea9/log.txt.gz 2026-03-10 05:16:30,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 05:16:31,041 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 05:16:31,043 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69af67af275c8d3b79399ea9/lava-logs.yaml 2026-03-10 05:16:31,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 05:16:31,168 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 05:16:31,169 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69af67af275c8d3b79399ea9/lava_callback.json.gz DEBUG: Setup failed for 69af67af275c8d3b79399ea9. Transitting job node result: incomplete -> fail 2026-03-10 05:16:31,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:16:31,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af67af275c8d3b79399ea9 HTTP/1.1" 200 None 2026-03-10 05:16:31,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:16:31,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af632c275c8d3b79399db4 HTTP/1.1" 200 None 2026-03-10 05:16:31,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:16:31,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af67af275c8d3b79399ea9 HTTP/1.1" 200 None 2026-03-10 05:16:31,339 - lava_callback - INFO - Completed processing callback for node 69af67af275c8d3b79399ea9 INFO: 172.20.0.1:46026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33148 - "GET / HTTP/1.0" 200 OK 2026-03-10 05:16:54,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:16:54,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:46602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50094 - "GET / HTTP/1.0" 200 OK 2026-03-10 05:17:10,725 - lava_callback - INFO - Received callback for node 69ad0650bcd207e915d78c00 INFO: 172.20.0.1:49346 - "POST /node/69ad0650bcd207e915d78c00 HTTP/1.0" 202 Accepted 2026-03-10 05:17:10,731 - lava_callback - INFO - Starting async job submit for node 69ad0650bcd207e915d78c00 2026-03-10 05:17:10,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:17:10,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad0650bcd207e915d78c00 HTTP/1.1" 200 None 2026-03-10 05:17:10,746 - lava_callback - DEBUG - Processing node 69ad0650bcd207e915d78c00: baseline-arm64-clabbe 2026-03-10 05:17:10,747 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ad0650bcd207e915d78c00 2026-03-10 05:17:10,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:17:10,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad0650bcd207e915d78c00 HTTP/1.1" 200 None 2026-03-10 05:17:10,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:17:10,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac4f7446f7ae8ff3364c45 HTTP/1.1" 200 None 2026-03-10 05:17:10,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:17:10,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ad0650bcd207e915d78c00 HTTP/1.1" 200 None 2026-03-10 05:17:10,810 - lava_callback - ERROR - Error processing callback for node 69ad0650bcd207e915d78c00: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49352 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53966 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43826 - "GET / HTTP/1.0" 200 OK 2026-03-10 05:18:30,758 - lava_callback - INFO - Received callback for node 69ad069dbcd207e915d78c26 INFO: 172.20.0.1:38804 - "POST /node/69ad069dbcd207e915d78c26 HTTP/1.0" 202 Accepted 2026-03-10 05:18:30,763 - lava_callback - INFO - Starting async job submit for node 69ad069dbcd207e915d78c26 2026-03-10 05:18:30,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:18:30,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad069dbcd207e915d78c26 HTTP/1.1" 200 None 2026-03-10 05:18:30,811 - lava_callback - DEBUG - Processing node 69ad069dbcd207e915d78c26: baseline-arm64-clabbe 2026-03-10 05:18:30,812 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ad069dbcd207e915d78c26 2026-03-10 05:18:30,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:18:30,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad069dbcd207e915d78c26 HTTP/1.1" 200 None 2026-03-10 05:18:30,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:18:30,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac4f7446f7ae8ff3364c45 HTTP/1.1" 200 None 2026-03-10 05:18:30,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:18:30,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ad069dbcd207e915d78c26 HTTP/1.1" 200 None 2026-03-10 05:18:30,873 - lava_callback - ERROR - Error processing callback for node 69ad069dbcd207e915d78c26: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52342 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45356 - "GET / HTTP/1.0" 200 OK 2026-03-10 05:19:30,820 - lava_callback - INFO - Received callback for node 69ad06dcbcd207e915d78c39 INFO: 172.20.0.1:43896 - "POST /node/69ad06dcbcd207e915d78c39 HTTP/1.0" 202 Accepted 2026-03-10 05:19:30,826 - lava_callback - INFO - Starting async job submit for node 69ad06dcbcd207e915d78c39 2026-03-10 05:19:30,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:19:30,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad06dcbcd207e915d78c39 HTTP/1.1" 200 None 2026-03-10 05:19:30,842 - lava_callback - DEBUG - Processing node 69ad06dcbcd207e915d78c39: baseline-arm-clabbe 2026-03-10 05:19:30,842 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ad06dcbcd207e915d78c39 2026-03-10 05:19:30,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:19:30,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad06dcbcd207e915d78c39 HTTP/1.1" 200 None 2026-03-10 05:19:30,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:19:30,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a9acd99c5f93e4f05eb2ac HTTP/1.1" 200 None 2026-03-10 05:19:30,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:19:30,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ad06dcbcd207e915d78c39 HTTP/1.1" 200 None 2026-03-10 05:19:30,906 - lava_callback - ERROR - Error processing callback for node 69ad06dcbcd207e915d78c39: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35824 - "GET / HTTP/1.0" 200 OK 2026-03-10 05:20:10,777 - lava_callback - INFO - Received callback for node 69ad0704bcd207e915d78c4e INFO: 172.20.0.1:49102 - "POST /node/69ad0704bcd207e915d78c4e HTTP/1.0" 202 Accepted 2026-03-10 05:20:10,782 - lava_callback - INFO - Starting async job submit for node 69ad0704bcd207e915d78c4e 2026-03-10 05:20:10,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:20:10,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad0704bcd207e915d78c4e HTTP/1.1" 200 None 2026-03-10 05:20:10,798 - lava_callback - DEBUG - Processing node 69ad0704bcd207e915d78c4e: baseline-arm-clabbe 2026-03-10 05:20:10,798 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ad0704bcd207e915d78c4e 2026-03-10 05:20:10,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:20:10,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad0704bcd207e915d78c4e HTTP/1.1" 200 None 2026-03-10 05:20:10,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:20:10,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a9acd99c5f93e4f05eb2ac HTTP/1.1" 200 None 2026-03-10 05:20:10,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:20:10,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ad0704bcd207e915d78c4e HTTP/1.1" 200 None 2026-03-10 05:20:10,853 - lava_callback - ERROR - Error processing callback for node 69ad0704bcd207e915d78c4e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49112 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41736 - "GET / HTTP/1.0" 200 OK 2026-03-10 05:21:10,899 - lava_callback - INFO - Received callback for node 69ad0739bcd207e915d78c71 INFO: 172.20.0.1:43298 - "POST /node/69ad0739bcd207e915d78c71 HTTP/1.0" 202 Accepted 2026-03-10 05:21:10,904 - lava_callback - INFO - Starting async job submit for node 69ad0739bcd207e915d78c71 2026-03-10 05:21:10,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:21:10,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad0739bcd207e915d78c71 HTTP/1.1" 200 None 2026-03-10 05:21:10,924 - lava_callback - DEBUG - Processing node 69ad0739bcd207e915d78c71: baseline-arm64-clabbe 2026-03-10 05:21:10,924 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ad0739bcd207e915d78c71 2026-03-10 05:21:10,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:21:10,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad0739bcd207e915d78c71 HTTP/1.1" 200 None 2026-03-10 05:21:10,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:21:10,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a9acf29c5f93e4f05eb2cc HTTP/1.1" 200 None 2026-03-10 05:21:10,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:21:10,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ad0739bcd207e915d78c71 HTTP/1.1" 200 None 2026-03-10 05:21:10,985 - lava_callback - ERROR - Error processing callback for node 69ad0739bcd207e915d78c71: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43308 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45090 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52604 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52606 - "GET / HTTP/1.0" 200 OK 2026-03-10 05:23:25,679 - lava_callback - INFO - Received callback for node 69ad07cbbcd207e915d78c9d INFO: 172.20.0.1:32770 - "POST /node/69ad07cbbcd207e915d78c9d HTTP/1.0" 202 Accepted 2026-03-10 05:23:25,684 - lava_callback - INFO - Starting async job submit for node 69ad07cbbcd207e915d78c9d 2026-03-10 05:23:25,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:23:25,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad07cbbcd207e915d78c9d HTTP/1.1" 200 None 2026-03-10 05:23:25,701 - lava_callback - DEBUG - Processing node 69ad07cbbcd207e915d78c9d: baseline-arm64-baylibre 2026-03-10 05:23:25,701 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ad07cbbcd207e915d78c9d 2026-03-10 05:23:25,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:23:25,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad07cbbcd207e915d78c9d HTTP/1.1" 200 None 2026-03-10 05:23:25,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:23:25,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a9acf29c5f93e4f05eb2cc HTTP/1.1" 200 None 2026-03-10 05:23:25,743 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:23:25,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ad07cbbcd207e915d78c9d HTTP/1.1" 200 None 2026-03-10 05:23:25,769 - lava_callback - ERROR - Error processing callback for node 69ad07cbbcd207e915d78c9d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:32784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33512 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52088 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44988 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42284 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47782 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41630 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53626 - "GET / HTTP/1.0" 200 OK 2026-03-10 05:29:51,373 - lava_callback - INFO - Received callback for node 69ad093cbcd207e915d78ceb INFO: 172.20.0.1:57982 - "POST /node/69ad093cbcd207e915d78ceb HTTP/1.0" 202 Accepted 2026-03-10 05:29:51,378 - lava_callback - INFO - Starting async job submit for node 69ad093cbcd207e915d78ceb 2026-03-10 05:29:51,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:29:51,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad093cbcd207e915d78ceb HTTP/1.1" 200 None 2026-03-10 05:29:51,404 - lava_callback - DEBUG - Processing node 69ad093cbcd207e915d78ceb: baseline-arm64-clabbe 2026-03-10 05:29:51,404 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ad093cbcd207e915d78ceb 2026-03-10 05:29:51,405 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:29:51,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad093cbcd207e915d78ceb HTTP/1.1" 200 None 2026-03-10 05:29:51,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:29:51,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a9acf29c5f93e4f05eb2cc HTTP/1.1" 200 None 2026-03-10 05:29:51,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:29:51,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ad093cbcd207e915d78ceb HTTP/1.1" 200 None 2026-03-10 05:29:51,456 - lava_callback - ERROR - Error processing callback for node 69ad093cbcd207e915d78ceb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33178 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33962 - "GET / HTTP/1.0" 200 OK 2026-03-10 05:31:11,396 - lava_callback - INFO - Received callback for node 69ad0989bcd207e915d78d0b INFO: 172.20.0.1:39306 - "POST /node/69ad0989bcd207e915d78d0b HTTP/1.0" 202 Accepted 2026-03-10 05:31:11,401 - lava_callback - INFO - Starting async job submit for node 69ad0989bcd207e915d78d0b 2026-03-10 05:31:11,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:31:11,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad0989bcd207e915d78d0b HTTP/1.1" 200 None 2026-03-10 05:31:11,417 - lava_callback - DEBUG - Processing node 69ad0989bcd207e915d78d0b: baseline-arm64-clabbe 2026-03-10 05:31:11,417 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ad0989bcd207e915d78d0b 2026-03-10 05:31:11,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:31:11,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad0989bcd207e915d78d0b HTTP/1.1" 200 None 2026-03-10 05:31:11,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:31:11,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a9acf29c5f93e4f05eb2cc HTTP/1.1" 200 None 2026-03-10 05:31:11,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:31:11,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ad0989bcd207e915d78d0b HTTP/1.1" 200 None 2026-03-10 05:31:11,487 - lava_callback - ERROR - Error processing callback for node 69ad0989bcd207e915d78d0b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39318 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49142 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35780 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49096 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52552 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56074 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37562 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38196 - "GET / HTTP/1.0" 200 OK 2026-03-10 05:37:30,875 - lava_callback - INFO - Received callback for node 69afa990275c8d3b7939b314 INFO: 172.20.0.1:48806 - "POST /node/69afa990275c8d3b7939b314 HTTP/1.0" 202 Accepted 2026-03-10 05:37:30,921 - lava_callback - INFO - Starting async job submit for node 69afa990275c8d3b7939b314 2026-03-10 05:37:30,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:37:30,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afa990275c8d3b7939b314 HTTP/1.1" 200 None 2026-03-10 05:37:30,961 - lava_callback - DEBUG - Processing node 69afa990275c8d3b7939b314: baseline-arm-pengutronix 2026-03-10 05:37:31,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 05:37:31,235 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 05:37:31,237 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69afa990275c8d3b7939b314/log.txt.gz 2026-03-10 05:37:31,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 05:37:31,615 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 05:37:31,617 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69afa990275c8d3b7939b314/lava-logs.yaml 2026-03-10 05:37:31,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 05:37:31,796 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 05:37:31,798 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69afa990275c8d3b7939b314/lava_callback.json.gz DEBUG: Setup failed for 69afa990275c8d3b7939b314. Transitting job node result: incomplete -> fail 2026-03-10 05:37:31,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:37:31,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afa990275c8d3b7939b314 HTTP/1.1" 200 None 2026-03-10 05:37:31,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:37:31,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af632c275c8d3b79399db4 HTTP/1.1" 200 None 2026-03-10 05:37:31,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:37:31,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afa990275c8d3b7939b314 HTTP/1.1" 200 None 2026-03-10 05:37:31,979 - lava_callback - INFO - Completed processing callback for node 69afa990275c8d3b7939b314 INFO: 172.20.0.1:48818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57418 - "GET / HTTP/1.0" 200 OK 2026-03-10 05:37:54,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 05:37:54,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:36984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50166 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40090 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54786 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35114 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49146 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33604 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54194 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56290 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59038 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46756 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57574 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38460 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49804 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33990 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55328 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42386 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52032 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52462 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47388 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43990 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43654 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41388 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34778 - "GET / HTTP/1.0" 200 OK 2026-03-10 06:00:12,185 - lava_callback - INFO - Received callback for node 69afaee5275c8d3b7939b3e9 INFO: 172.20.0.1:42710 - "POST /node/69afaee5275c8d3b7939b3e9 HTTP/1.0" 202 Accepted 2026-03-10 06:00:12,233 - lava_callback - INFO - Starting async job submit for node 69afaee5275c8d3b7939b3e9 2026-03-10 06:00:12,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:00:12,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afaee5275c8d3b7939b3e9 HTTP/1.1" 200 None 2026-03-10 06:00:12,279 - lava_callback - DEBUG - Processing node 69afaee5275c8d3b7939b3e9: baseline-arm-pengutronix 2026-03-10 06:00:12,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 06:00:12,547 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 06:00:12,549 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69afaee5275c8d3b7939b3e9/log.txt.gz 2026-03-10 06:00:12,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 06:00:12,950 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 06:00:12,952 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69afaee5275c8d3b7939b3e9/lava-logs.yaml 2026-03-10 06:00:12,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 06:00:13,075 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 06:00:13,077 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69afaee5275c8d3b7939b3e9/lava_callback.json.gz DEBUG: Setup failed for 69afaee5275c8d3b7939b3e9. Transitting job node result: incomplete -> fail 2026-03-10 06:00:13,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:00:13,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afaee5275c8d3b7939b3e9 HTTP/1.1" 200 None 2026-03-10 06:00:13,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:00:13,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af632c275c8d3b79399db4 HTTP/1.1" 200 None 2026-03-10 06:00:13,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:00:13,195 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afaee5275c8d3b7939b3e9 HTTP/1.1" 200 None 2026-03-10 06:00:13,239 - lava_callback - INFO - Completed processing callback for node 69afaee5275c8d3b7939b3e9 INFO: 172.20.0.1:42718 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42726 - "GET / HTTP/1.0" 200 OK 2026-03-10 06:00:24,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:00:24,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:37852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54236 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37046 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45458 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51470 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58268 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38254 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58052 - "GET /node/6994465e5ae459b9df3d9f76%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:58068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51898 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35862 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55132 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34480 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47170 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49048 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35626 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40986 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53522 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36070 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50210 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59482 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43760 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59234 - "GET / HTTP/1.0" 200 OK 2026-03-10 06:19:53,603 - lava_callback - INFO - Received callback for node 69afb37b275c8d3b7939b467 INFO: 172.20.0.1:47750 - "POST /node/69afb37b275c8d3b7939b467 HTTP/1.0" 202 Accepted 2026-03-10 06:19:53,649 - lava_callback - INFO - Starting async job submit for node 69afb37b275c8d3b7939b467 2026-03-10 06:19:53,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:19:53,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afb37b275c8d3b7939b467 HTTP/1.1" 200 None 2026-03-10 06:19:53,710 - lava_callback - DEBUG - Processing node 69afb37b275c8d3b7939b467: baseline-arm-pengutronix 2026-03-10 06:19:53,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 06:19:53,963 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 06:19:53,964 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69afb37b275c8d3b7939b467/log.txt.gz 2026-03-10 06:19:54,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 06:19:54,208 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 06:19:54,210 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69afb37b275c8d3b7939b467/lava-logs.yaml 2026-03-10 06:19:54,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 06:19:54,356 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 06:19:54,358 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69afb37b275c8d3b7939b467/lava_callback.json.gz DEBUG: Setup failed for 69afb37b275c8d3b7939b467. Transitting job node result: incomplete -> fail 2026-03-10 06:19:54,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:19:54,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afb37b275c8d3b7939b467 HTTP/1.1" 200 None 2026-03-10 06:19:54,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:19:54,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af632c275c8d3b79399db4 HTTP/1.1" 200 None 2026-03-10 06:19:54,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:19:54,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afb37b275c8d3b7939b467 HTTP/1.1" 200 None 2026-03-10 06:19:54,521 - lava_callback - INFO - Completed processing callback for node 69afb37b275c8d3b7939b467 2026-03-10 06:19:54,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:19:54,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:47754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40892 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54494 - "GET / HTTP/1.0" 200 OK 2026-03-10 06:20:39,434 - lava_callback - INFO - Received callback for node 69aec8a7feb96b28aa34b72c INFO: 172.20.0.1:54502 - "POST /node/69aec8a7feb96b28aa34b72c HTTP/1.0" 202 Accepted 2026-03-10 06:20:39,441 - lava_callback - INFO - Starting async job submit for node 69aec8a7feb96b28aa34b72c 2026-03-10 06:20:39,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:20:39,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aec8a7feb96b28aa34b72c HTTP/1.1" 200 None 2026-03-10 06:20:39,492 - lava_callback - DEBUG - Processing node 69aec8a7feb96b28aa34b72c: baseline-arm64-broonie 2026-03-10 06:20:39,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 06:20:40,017 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 06:20:40,018 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69aec8a7feb96b28aa34b72c/log.txt.gz 2026-03-10 06:20:40,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 06:20:40,423 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 06:20:40,425 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69aec8a7feb96b28aa34b72c/lava-logs.yaml 2026-03-10 06:20:40,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 06:20:40,595 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 06:20:40,597 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69aec8a7feb96b28aa34b72c/lava_callback.json.gz DEBUG: Setup failed for 69aec8a7feb96b28aa34b72c. Transitting job node result: incomplete -> fail 2026-03-10 06:20:40,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:20:40,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aec8a7feb96b28aa34b72c HTTP/1.1" 200 None 2026-03-10 06:20:40,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:20:40,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aeba41feb96b28aa34a78b HTTP/1.1" 200 None 2026-03-10 06:20:40,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:20:40,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aec8a7feb96b28aa34b72c HTTP/1.1" 200 None 2026-03-10 06:20:40,793 - lava_callback - INFO - Completed processing callback for node 69aec8a7feb96b28aa34b72c INFO: 172.20.0.1:59616 - "GET / HTTP/1.0" 200 OK 2026-03-10 06:20:54,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:20:54,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:50552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53686 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53606 - "GET / HTTP/1.0" 200 OK 2026-03-10 06:21:32,996 - lava_callback - INFO - Received callback for node 69af25a6ce46d8ec186acfbb INFO: 172.20.0.1:34224 - "POST /node/69af25a6ce46d8ec186acfbb HTTP/1.0" 202 Accepted 2026-03-10 06:21:33,003 - lava_callback - INFO - Starting async job submit for node 69af25a6ce46d8ec186acfbb 2026-03-10 06:21:33,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:21:33,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af25a6ce46d8ec186acfbb HTTP/1.1" 200 None 2026-03-10 06:21:33,062 - lava_callback - DEBUG - Processing node 69af25a6ce46d8ec186acfbb: baseline-arm64-broonie 2026-03-10 06:21:33,334 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 06:21:33,488 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 06:21:33,490 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af25a6ce46d8ec186acfbb/log.txt.gz 2026-03-10 06:21:33,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 06:21:33,842 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 06:21:33,845 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af25a6ce46d8ec186acfbb/lava-logs.yaml 2026-03-10 06:21:33,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 06:21:34,016 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 06:21:34,018 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af25a6ce46d8ec186acfbb/lava_callback.json.gz DEBUG: Setup failed for 69af25a6ce46d8ec186acfbb. Transitting job node result: incomplete -> fail 2026-03-10 06:21:34,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:21:34,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af25a6ce46d8ec186acfbb HTTP/1.1" 200 None 2026-03-10 06:21:34,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:21:34,129 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-10 06:21:34,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:21:34,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af25a6ce46d8ec186acfbb HTTP/1.1" 200 None 2026-03-10 06:21:34,228 - lava_callback - INFO - Completed processing callback for node 69af25a6ce46d8ec186acfbb INFO: 172.20.0.1:34240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38210 - "GET / HTTP/1.0" 200 OK 2026-03-10 06:21:54,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:21:54,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:54532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46408 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50056 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46250 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40236 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43202 - "GET / HTTP/1.0" 200 OK 2026-03-10 06:25:32,743 - lava_callback - INFO - Received callback for node 69af6c08275c8d3b7939a542 INFO: 172.20.0.1:42066 - "POST /node/69af6c08275c8d3b7939a542 HTTP/1.0" 202 Accepted 2026-03-10 06:25:32,749 - lava_callback - INFO - Starting async job submit for node 69af6c08275c8d3b7939a542 2026-03-10 06:25:32,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:25:32,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6c08275c8d3b7939a542 HTTP/1.1" 200 None 2026-03-10 06:25:32,836 - lava_callback - DEBUG - Processing node 69af6c08275c8d3b7939a542: baseline-arm64-broonie 2026-03-10 06:25:33,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 06:25:33,232 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 06:25:33,234 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af6c08275c8d3b7939a542/log.txt.gz 2026-03-10 06:25:33,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 06:25:33,596 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 06:25:33,598 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af6c08275c8d3b7939a542/lava-logs.yaml 2026-03-10 06:25:33,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 06:25:33,784 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 06:25:33,786 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af6c08275c8d3b7939a542/lava_callback.json.gz DEBUG: Setup failed for 69af6c08275c8d3b7939a542. Transitting job node result: incomplete -> fail 2026-03-10 06:25:33,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:25:33,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6c08275c8d3b7939a542 HTTP/1.1" 200 None 2026-03-10 06:25:33,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:25:33,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 06:25:33,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:25:33,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af6c08275c8d3b7939a542 HTTP/1.1" 200 None 2026-03-10 06:25:34,037 - lava_callback - INFO - Completed processing callback for node 69af6c08275c8d3b7939a542 INFO: 172.20.0.1:42074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52836 - "GET / HTTP/1.0" 200 OK 2026-03-10 06:25:54,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:25:54,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:41544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44412 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52910 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50732 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44732 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44736 - "GET / HTTP/1.0" 200 OK 2026-03-10 06:29:19,696 - lava_callback - INFO - Received callback for node 69af01c2ce46d8ec186ac6cd INFO: 172.20.0.1:49144 - "POST /node/69af01c2ce46d8ec186ac6cd HTTP/1.0" 202 Accepted 2026-03-10 06:29:19,702 - lava_callback - INFO - Starting async job submit for node 69af01c2ce46d8ec186ac6cd 2026-03-10 06:29:19,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:29:19,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01c2ce46d8ec186ac6cd HTTP/1.1" 200 None 2026-03-10 06:29:19,755 - lava_callback - DEBUG - Processing node 69af01c2ce46d8ec186ac6cd: baseline-arm64-broonie 2026-03-10 06:29:20,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 06:29:20,252 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 06:29:20,254 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af01c2ce46d8ec186ac6cd/log.txt.gz 2026-03-10 06:29:20,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 06:29:20,739 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 06:29:20,741 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af01c2ce46d8ec186ac6cd/lava-logs.yaml 2026-03-10 06:29:20,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 06:29:20,964 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 06:29:20,966 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af01c2ce46d8ec186ac6cd/lava_callback.json.gz DEBUG: Setup failed for 69af01c2ce46d8ec186ac6cd. Transitting job node result: incomplete -> fail 2026-03-10 06:29:21,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:29:21,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01c2ce46d8ec186ac6cd HTTP/1.1" 200 None 2026-03-10 06:29:21,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:29:21,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-10 06:29:21,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:29:21,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af01c2ce46d8ec186ac6cd HTTP/1.1" 200 None 2026-03-10 06:29:21,189 - lava_callback - INFO - Completed processing callback for node 69af01c2ce46d8ec186ac6cd 2026-03-10 06:29:24,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:29:24,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:49146 - "GET / HTTP/1.0" 200 OK 2026-03-10 06:29:33,334 - lava_callback - INFO - Received callback for node 69af8a9b275c8d3b7939acf7 INFO: 172.20.0.1:33288 - "POST /node/69af8a9b275c8d3b7939acf7 HTTP/1.0" 202 Accepted 2026-03-10 06:29:33,340 - lava_callback - INFO - Starting async job submit for node 69af8a9b275c8d3b7939acf7 2026-03-10 06:29:33,376 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:29:33,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af8a9b275c8d3b7939acf7 HTTP/1.1" 200 None 2026-03-10 06:29:33,392 - lava_callback - DEBUG - Processing node 69af8a9b275c8d3b7939acf7: baseline-arm64-broonie 2026-03-10 06:29:33,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 06:29:33,759 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 06:29:33,761 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af8a9b275c8d3b7939acf7/log.txt.gz 2026-03-10 06:29:33,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 06:29:34,091 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 06:29:34,093 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af8a9b275c8d3b7939acf7/lava-logs.yaml 2026-03-10 06:29:34,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 06:29:34,282 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 06:29:34,284 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af8a9b275c8d3b7939acf7/lava_callback.json.gz DEBUG: Setup failed for 69af8a9b275c8d3b7939acf7. Transitting job node result: incomplete -> fail 2026-03-10 06:29:34,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:29:34,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af8a9b275c8d3b7939acf7 HTTP/1.1" 200 None 2026-03-10 06:29:34,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:29:34,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-10 06:29:34,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:29:34,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af8a9b275c8d3b7939acf7 HTTP/1.1" 200 None 2026-03-10 06:29:34,505 - lava_callback - INFO - Completed processing callback for node 69af8a9b275c8d3b7939acf7 INFO: 172.20.0.1:33294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58444 - "GET / HTTP/1.0" 200 OK 2026-03-10 06:29:54,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:29:54,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:50216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47598 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60086 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44886 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43772 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59094 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42600 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45058 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51214 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54610 - "GET / HTTP/1.0" 200 OK 2026-03-10 06:38:00,083 - lava_callback - INFO - Received callback for node 69af3414ce46d8ec186ad276 INFO: 172.20.0.1:44112 - "POST /node/69af3414ce46d8ec186ad276 HTTP/1.0" 202 Accepted 2026-03-10 06:38:00,089 - lava_callback - INFO - Starting async job submit for node 69af3414ce46d8ec186ad276 2026-03-10 06:38:00,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:38:00,145 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af3414ce46d8ec186ad276 HTTP/1.1" 200 None 2026-03-10 06:38:00,146 - lava_callback - DEBUG - Processing node 69af3414ce46d8ec186ad276: baseline-arm64-broonie 2026-03-10 06:38:00,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 06:38:00,668 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 06:38:00,670 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af3414ce46d8ec186ad276/log.txt.gz 2026-03-10 06:38:00,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 06:38:01,108 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 06:38:01,110 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af3414ce46d8ec186ad276/lava-logs.yaml 2026-03-10 06:38:01,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 06:38:01,278 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 06:38:01,280 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af3414ce46d8ec186ad276/lava_callback.json.gz DEBUG: Setup failed for 69af3414ce46d8ec186ad276. Transitting job node result: incomplete -> fail 2026-03-10 06:38:01,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:38:01,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af3414ce46d8ec186ad276 HTTP/1.1" 200 None 2026-03-10 06:38:01,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:38:01,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aeba41feb96b28aa34a78b HTTP/1.1" 200 None 2026-03-10 06:38:01,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:38:01,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af3414ce46d8ec186ad276 HTTP/1.1" 200 None 2026-03-10 06:38:01,502 - lava_callback - INFO - Completed processing callback for node 69af3414ce46d8ec186ad276 INFO: 172.20.0.1:44116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46892 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46896 - "GET / HTTP/1.0" 200 OK 2026-03-10 06:38:24,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:38:24,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:37208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34820 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40716 - "GET / HTTP/1.0" 200 OK 2026-03-10 06:39:34,479 - lava_callback - INFO - Received callback for node 69ad199abcd207e915d78ff7 INFO: 172.20.0.1:58356 - "POST /node/69ad199abcd207e915d78ff7 HTTP/1.0" 202 Accepted 2026-03-10 06:39:34,484 - lava_callback - INFO - Starting async job submit for node 69ad199abcd207e915d78ff7 2026-03-10 06:39:34,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:39:34,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad199abcd207e915d78ff7 HTTP/1.1" 200 None 2026-03-10 06:39:34,505 - lava_callback - DEBUG - Processing node 69ad199abcd207e915d78ff7: baseline-arm-clabbe 2026-03-10 06:39:34,505 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ad199abcd207e915d78ff7 2026-03-10 06:39:34,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:39:34,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad199abcd207e915d78ff7 HTTP/1.1" 200 None 2026-03-10 06:39:34,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:39:34,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa1d368c4d487a0e751145 HTTP/1.1" 200 None 2026-03-10 06:39:34,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:39:34,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ad199abcd207e915d78ff7 HTTP/1.1" 200 None 2026-03-10 06:39:34,560 - lava_callback - ERROR - Error processing callback for node 69ad199abcd207e915d78ff7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57578 - "GET / HTTP/1.0" 200 OK 2026-03-10 06:40:14,396 - lava_callback - INFO - Received callback for node 69ad19c2bcd207e915d79005 INFO: 172.20.0.1:59818 - "POST /node/69ad19c2bcd207e915d79005 HTTP/1.0" 202 Accepted 2026-03-10 06:40:14,401 - lava_callback - INFO - Starting async job submit for node 69ad19c2bcd207e915d79005 2026-03-10 06:40:14,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:40:14,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad19c2bcd207e915d79005 HTTP/1.1" 200 None 2026-03-10 06:40:14,420 - lava_callback - DEBUG - Processing node 69ad19c2bcd207e915d79005: baseline-arm-clabbe 2026-03-10 06:40:14,420 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ad19c2bcd207e915d79005 2026-03-10 06:40:14,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:40:14,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad19c2bcd207e915d79005 HTTP/1.1" 200 None 2026-03-10 06:40:14,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:40:14,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa1d368c4d487a0e751145 HTTP/1.1" 200 None 2026-03-10 06:40:14,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:40:14,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ad19c2bcd207e915d79005 HTTP/1.1" 200 None 2026-03-10 06:40:14,478 - lava_callback - ERROR - Error processing callback for node 69ad19c2bcd207e915d79005: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59826 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38304 - "GET / HTTP/1.0" 200 OK 2026-03-10 06:40:54,670 - lava_callback - INFO - Received callback for node 69ad19eabcd207e915d79021 INFO: 172.20.0.1:35358 - "POST /node/69ad19eabcd207e915d79021 HTTP/1.0" 202 Accepted 2026-03-10 06:40:54,676 - lava_callback - INFO - Starting async job submit for node 69ad19eabcd207e915d79021 2026-03-10 06:40:54,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:40:54,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad19eabcd207e915d79021 HTTP/1.1" 200 None 2026-03-10 06:40:54,692 - lava_callback - DEBUG - Processing node 69ad19eabcd207e915d79021: baseline-arm-clabbe 2026-03-10 06:40:54,692 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ad19eabcd207e915d79021 2026-03-10 06:40:54,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:40:54,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad19eabcd207e915d79021 HTTP/1.1" 200 None 2026-03-10 06:40:54,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:40:54,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa1d368c4d487a0e751145 HTTP/1.1" 200 None 2026-03-10 06:40:54,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:40:54,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ad19eabcd207e915d79021 HTTP/1.1" 200 None 2026-03-10 06:40:54,746 - lava_callback - ERROR - Error processing callback for node 69ad19eabcd207e915d79021: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36958 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42830 - "GET / HTTP/1.0" 200 OK 2026-03-10 06:41:34,466 - lava_callback - INFO - Received callback for node 69ad1a10bcd207e915d79048 INFO: 172.20.0.1:36670 - "POST /node/69ad1a10bcd207e915d79048 HTTP/1.0" 202 Accepted 2026-03-10 06:41:34,471 - lava_callback - INFO - Starting async job submit for node 69ad1a10bcd207e915d79048 2026-03-10 06:41:34,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:41:34,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad1a10bcd207e915d79048 HTTP/1.1" 200 None 2026-03-10 06:41:34,490 - lava_callback - DEBUG - Processing node 69ad1a10bcd207e915d79048: baseline-arm-clabbe 2026-03-10 06:41:34,490 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ad1a10bcd207e915d79048 2026-03-10 06:41:34,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:41:34,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad1a10bcd207e915d79048 HTTP/1.1" 200 None 2026-03-10 06:41:34,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:41:34,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa1d368c4d487a0e751145 HTTP/1.1" 200 None 2026-03-10 06:41:34,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:41:34,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ad1a10bcd207e915d79048 HTTP/1.1" 200 None 2026-03-10 06:41:34,547 - lava_callback - ERROR - Error processing callback for node 69ad1a10bcd207e915d79048: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51570 - "GET / HTTP/1.0" 200 OK 2026-03-10 06:42:14,540 - lava_callback - INFO - Received callback for node 69ad1a38bcd207e915d79065 INFO: 172.20.0.1:54340 - "POST /node/69ad1a38bcd207e915d79065 HTTP/1.0" 202 Accepted 2026-03-10 06:42:14,546 - lava_callback - INFO - Starting async job submit for node 69ad1a38bcd207e915d79065 2026-03-10 06:42:14,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:42:14,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad1a38bcd207e915d79065 HTTP/1.1" 200 None 2026-03-10 06:42:14,562 - lava_callback - DEBUG - Processing node 69ad1a38bcd207e915d79065: baseline-arm-clabbe 2026-03-10 06:42:14,562 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ad1a38bcd207e915d79065 2026-03-10 06:42:14,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:42:14,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad1a38bcd207e915d79065 HTTP/1.1" 200 None 2026-03-10 06:42:14,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:42:14,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa1d368c4d487a0e751145 HTTP/1.1" 200 None 2026-03-10 06:42:14,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:42:14,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ad1a38bcd207e915d79065 HTTP/1.1" 200 None 2026-03-10 06:42:14,637 - lava_callback - ERROR - Error processing callback for node 69ad1a38bcd207e915d79065: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54350 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36242 - "GET / HTTP/1.0" 200 OK 2026-03-10 06:42:54,527 - lava_callback - INFO - Received callback for node 69ad1a5ebcd207e915d79076 INFO: 172.20.0.1:52390 - "POST /node/69ad1a5ebcd207e915d79076 HTTP/1.0" 202 Accepted 2026-03-10 06:42:54,536 - lava_callback - INFO - Starting async job submit for node 69ad1a5ebcd207e915d79076 2026-03-10 06:42:54,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:42:54,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad1a5ebcd207e915d79076 HTTP/1.1" 200 None 2026-03-10 06:42:54,552 - lava_callback - DEBUG - Processing node 69ad1a5ebcd207e915d79076: baseline-arm-clabbe 2026-03-10 06:42:54,552 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ad1a5ebcd207e915d79076 2026-03-10 06:42:54,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:42:54,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad1a5ebcd207e915d79076 HTTP/1.1" 200 None 2026-03-10 06:42:54,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:42:54,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa1d368c4d487a0e751145 HTTP/1.1" 200 None 2026-03-10 06:42:54,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:42:54,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ad1a5ebcd207e915d79076 HTTP/1.1" 200 None 2026-03-10 06:42:54,608 - lava_callback - ERROR - Error processing callback for node 69ad1a5ebcd207e915d79076: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43214 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49402 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53896 - "GET / HTTP/1.0" 200 OK 2026-03-10 06:44:30,076 - lava_callback - INFO - Received callback for node 69ad1abebcd207e915d7909e INFO: 172.20.0.1:42534 - "POST /node/69ad1abebcd207e915d7909e HTTP/1.0" 202 Accepted 2026-03-10 06:44:30,082 - lava_callback - INFO - Starting async job submit for node 69ad1abebcd207e915d7909e 2026-03-10 06:44:30,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:44:30,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad1abebcd207e915d7909e HTTP/1.1" 200 None 2026-03-10 06:44:30,099 - lava_callback - DEBUG - Processing node 69ad1abebcd207e915d7909e: baseline-arm-baylibre 2026-03-10 06:44:30,099 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ad1abebcd207e915d7909e 2026-03-10 06:44:30,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:44:30,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad1abebcd207e915d7909e HTTP/1.1" 200 None 2026-03-10 06:44:30,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:44:30,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a77a57a255cc0324eb010a HTTP/1.1" 200 None 2026-03-10 06:44:30,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:44:30,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ad1abebcd207e915d7909e HTTP/1.1" 200 None 2026-03-10 06:44:30,171 - lava_callback - ERROR - Error processing callback for node 69ad1abebcd207e915d7909e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59136 - "GET / HTTP/1.0" 200 OK 2026-03-10 06:45:10,050 - lava_callback - INFO - Received callback for node 69ad1ae6bcd207e915d790b3 INFO: 172.20.0.1:51434 - "POST /node/69ad1ae6bcd207e915d790b3 HTTP/1.0" 202 Accepted 2026-03-10 06:45:10,056 - lava_callback - INFO - Starting async job submit for node 69ad1ae6bcd207e915d790b3 2026-03-10 06:45:10,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:45:10,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad1ae6bcd207e915d790b3 HTTP/1.1" 200 None 2026-03-10 06:45:10,072 - lava_callback - DEBUG - Processing node 69ad1ae6bcd207e915d790b3: baseline-arm-baylibre 2026-03-10 06:45:10,073 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ad1ae6bcd207e915d790b3 2026-03-10 06:45:10,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:45:10,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad1ae6bcd207e915d790b3 HTTP/1.1" 200 None 2026-03-10 06:45:10,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:45:10,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a77a57a255cc0324eb010a HTTP/1.1" 200 None 2026-03-10 06:45:10,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:45:10,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ad1ae6bcd207e915d790b3 HTTP/1.1" 200 None 2026-03-10 06:45:10,140 - lava_callback - ERROR - Error processing callback for node 69ad1ae6bcd207e915d790b3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51446 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40188 - "GET / HTTP/1.0" 200 OK 2026-03-10 06:45:50,121 - lava_callback - INFO - Received callback for node 69ad1b0fbcd207e915d790c5 INFO: 172.20.0.1:44552 - "POST /node/69ad1b0fbcd207e915d790c5 HTTP/1.0" 202 Accepted 2026-03-10 06:45:50,126 - lava_callback - INFO - Starting async job submit for node 69ad1b0fbcd207e915d790c5 2026-03-10 06:45:50,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:45:50,149 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad1b0fbcd207e915d790c5 HTTP/1.1" 200 None 2026-03-10 06:45:50,150 - lava_callback - DEBUG - Processing node 69ad1b0fbcd207e915d790c5: baseline-arm-baylibre 2026-03-10 06:45:50,151 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ad1b0fbcd207e915d790c5 2026-03-10 06:45:50,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:45:50,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad1b0fbcd207e915d790c5 HTTP/1.1" 200 None 2026-03-10 06:45:50,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:45:50,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a77a57a255cc0324eb010a HTTP/1.1" 200 None 2026-03-10 06:45:50,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:45:50,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ad1b0fbcd207e915d790c5 HTTP/1.1" 200 None 2026-03-10 06:45:50,215 - lava_callback - ERROR - Error processing callback for node 69ad1b0fbcd207e915d790c5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51936 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49646 - "GET / HTTP/1.0" 200 OK 2026-03-10 06:46:30,090 - lava_callback - INFO - Received callback for node 69ad1b37bcd207e915d790da INFO: 172.20.0.1:60800 - "POST /node/69ad1b37bcd207e915d790da HTTP/1.0" 202 Accepted 2026-03-10 06:46:30,097 - lava_callback - INFO - Starting async job submit for node 69ad1b37bcd207e915d790da 2026-03-10 06:46:30,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:46:30,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad1b37bcd207e915d790da HTTP/1.1" 200 None 2026-03-10 06:46:30,116 - lava_callback - DEBUG - Processing node 69ad1b37bcd207e915d790da: baseline-arm-baylibre 2026-03-10 06:46:30,117 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ad1b37bcd207e915d790da 2026-03-10 06:46:30,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:46:30,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad1b37bcd207e915d790da HTTP/1.1" 200 None 2026-03-10 06:46:30,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:46:30,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a77a57a255cc0324eb010a HTTP/1.1" 200 None 2026-03-10 06:46:30,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:46:30,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ad1b37bcd207e915d790da HTTP/1.1" 200 None 2026-03-10 06:46:30,183 - lava_callback - ERROR - Error processing callback for node 69ad1b37bcd207e915d790da: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60812 - "GET / HTTP/1.0" 200 OK 2026-03-10 06:46:40,443 - lava_callback - INFO - Received callback for node 69af5a71ce46d8ec186ad8a9 INFO: 172.20.0.1:32890 - "POST /node/69af5a71ce46d8ec186ad8a9 HTTP/1.0" 202 Accepted 2026-03-10 06:46:40,449 - lava_callback - INFO - Starting async job submit for node 69af5a71ce46d8ec186ad8a9 2026-03-10 06:46:40,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:46:40,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af5a71ce46d8ec186ad8a9 HTTP/1.1" 200 None 2026-03-10 06:46:40,500 - lava_callback - DEBUG - Processing node 69af5a71ce46d8ec186ad8a9: baseline-arm64-broonie 2026-03-10 06:46:40,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 06:46:41,169 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 06:46:41,171 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af5a71ce46d8ec186ad8a9/log.txt.gz 2026-03-10 06:46:41,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 06:46:41,625 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 06:46:41,627 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af5a71ce46d8ec186ad8a9/lava-logs.yaml 2026-03-10 06:46:41,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 06:46:41,816 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 06:46:41,818 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af5a71ce46d8ec186ad8a9/lava_callback.json.gz DEBUG: Setup failed for 69af5a71ce46d8ec186ad8a9. Transitting job node result: incomplete -> fail 2026-03-10 06:46:41,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:46:41,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af5a71ce46d8ec186ad8a9 HTTP/1.1" 200 None 2026-03-10 06:46:41,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:46:41,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-10 06:46:41,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:46:41,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af5a71ce46d8ec186ad8a9 HTTP/1.1" 200 None 2026-03-10 06:46:42,040 - lava_callback - INFO - Completed processing callback for node 69af5a71ce46d8ec186ad8a9 INFO: 172.20.0.1:32894 - "GET / HTTP/1.0" 200 OK 2026-03-10 06:46:54,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:46:54,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:48132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54366 - "GET / HTTP/1.0" 200 OK 2026-03-10 06:47:10,191 - lava_callback - INFO - Received callback for node 69ad1b5fbcd207e915d790fa INFO: 172.20.0.1:50292 - "POST /node/69ad1b5fbcd207e915d790fa HTTP/1.0" 202 Accepted 2026-03-10 06:47:10,196 - lava_callback - INFO - Starting async job submit for node 69ad1b5fbcd207e915d790fa 2026-03-10 06:47:10,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:47:10,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad1b5fbcd207e915d790fa HTTP/1.1" 200 None 2026-03-10 06:47:10,215 - lava_callback - DEBUG - Processing node 69ad1b5fbcd207e915d790fa: baseline-arm-baylibre 2026-03-10 06:47:10,216 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ad1b5fbcd207e915d790fa 2026-03-10 06:47:10,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:47:10,231 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad1b5fbcd207e915d790fa HTTP/1.1" 200 None 2026-03-10 06:47:10,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:47:10,248 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a77a57a255cc0324eb010a HTTP/1.1" 200 None 2026-03-10 06:47:10,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:47:10,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ad1b5fbcd207e915d790fa HTTP/1.1" 200 None 2026-03-10 06:47:10,274 - lava_callback - ERROR - Error processing callback for node 69ad1b5fbcd207e915d790fa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50308 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55996 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33538 - "GET / HTTP/1.0" 200 OK 2026-03-10 06:48:30,252 - lava_callback - INFO - Received callback for node 69ad1babbcd207e915d79126 INFO: 172.20.0.1:58120 - "POST /node/69ad1babbcd207e915d79126 HTTP/1.0" 202 Accepted 2026-03-10 06:48:30,258 - lava_callback - INFO - Starting async job submit for node 69ad1babbcd207e915d79126 2026-03-10 06:48:30,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:48:30,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad1babbcd207e915d79126 HTTP/1.1" 200 None 2026-03-10 06:48:30,285 - lava_callback - DEBUG - Processing node 69ad1babbcd207e915d79126: baseline-arm-baylibre 2026-03-10 06:48:30,285 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ad1babbcd207e915d79126 2026-03-10 06:48:30,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:48:30,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad1babbcd207e915d79126 HTTP/1.1" 200 None 2026-03-10 06:48:30,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:48:30,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a77a57a255cc0324eb010a HTTP/1.1" 200 None 2026-03-10 06:48:30,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:48:30,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ad1babbcd207e915d79126 HTTP/1.1" 200 None 2026-03-10 06:48:30,343 - lava_callback - ERROR - Error processing callback for node 69ad1babbcd207e915d79126: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36958 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50536 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55632 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54578 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58624 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59882 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60860 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60876 - "GET / HTTP/1.0" 200 OK 2026-03-10 06:55:20,769 - lava_callback - INFO - Received callback for node 69af6c03275c8d3b7939a533 INFO: 172.20.0.1:56372 - "POST /node/69af6c03275c8d3b7939a533 HTTP/1.0" 202 Accepted 2026-03-10 06:55:20,776 - lava_callback - INFO - Starting async job submit for node 69af6c03275c8d3b7939a533 2026-03-10 06:55:20,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:55:20,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6c03275c8d3b7939a533 HTTP/1.1" 200 None 2026-03-10 06:55:20,829 - lava_callback - DEBUG - Processing node 69af6c03275c8d3b7939a533: baseline-arm64-broonie 2026-03-10 06:55:21,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 06:55:21,328 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 06:55:21,330 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af6c03275c8d3b7939a533/log.txt.gz 2026-03-10 06:55:21,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 06:55:21,779 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 06:55:21,781 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af6c03275c8d3b7939a533/lava-logs.yaml 2026-03-10 06:55:21,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 06:55:21,939 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 06:55:21,940 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af6c03275c8d3b7939a533/lava_callback.json.gz DEBUG: Setup failed for 69af6c03275c8d3b7939a533. Transitting job node result: incomplete -> fail 2026-03-10 06:55:22,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:55:22,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6c03275c8d3b7939a533 HTTP/1.1" 200 None 2026-03-10 06:55:22,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:55:22,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 06:55:22,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:55:22,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af6c03275c8d3b7939a533 HTTP/1.1" 200 None 2026-03-10 06:55:22,162 - lava_callback - INFO - Completed processing callback for node 69af6c03275c8d3b7939a533 2026-03-10 06:55:24,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:55:24,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:56386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37724 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33540 - "GET / HTTP/1.0" 200 OK 2026-03-10 06:56:55,504 - lava_callback - INFO - Received callback for node 69ad1da3bcd207e915d7923a INFO: 172.20.0.1:50652 - "POST /node/69ad1da3bcd207e915d7923a HTTP/1.0" 202 Accepted 2026-03-10 06:56:55,510 - lava_callback - INFO - Starting async job submit for node 69ad1da3bcd207e915d7923a 2026-03-10 06:56:55,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:56:55,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad1da3bcd207e915d7923a HTTP/1.1" 200 None 2026-03-10 06:56:55,536 - lava_callback - DEBUG - Processing node 69ad1da3bcd207e915d7923a: baseline-arm-clabbe 2026-03-10 06:56:55,536 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ad1da3bcd207e915d7923a 2026-03-10 06:56:55,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:56:55,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad1da3bcd207e915d7923a HTTP/1.1" 200 None 2026-03-10 06:56:55,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:56:55,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acc009bcd207e915d76b94 HTTP/1.1" 200 None 2026-03-10 06:56:55,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:56:55,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ad1da3bcd207e915d7923a HTTP/1.1" 200 None 2026-03-10 06:56:55,617 - lava_callback - ERROR - Error processing callback for node 69ad1da3bcd207e915d7923a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54610 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59684 - "GET / HTTP/1.0" 200 OK 2026-03-10 06:57:35,244 - lava_callback - INFO - Received callback for node 69ad1dc9bcd207e915d7924b INFO: 172.20.0.1:43674 - "POST /node/69ad1dc9bcd207e915d7924b HTTP/1.0" 202 Accepted 2026-03-10 06:57:35,249 - lava_callback - INFO - Starting async job submit for node 69ad1dc9bcd207e915d7924b 2026-03-10 06:57:35,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:57:35,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad1dc9bcd207e915d7924b HTTP/1.1" 200 None 2026-03-10 06:57:35,266 - lava_callback - DEBUG - Processing node 69ad1dc9bcd207e915d7924b: baseline-arm-clabbe 2026-03-10 06:57:35,266 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ad1dc9bcd207e915d7924b 2026-03-10 06:57:35,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:57:35,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad1dc9bcd207e915d7924b HTTP/1.1" 200 None 2026-03-10 06:57:35,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:57:35,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acc009bcd207e915d76b94 HTTP/1.1" 200 None 2026-03-10 06:57:35,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:57:35,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ad1dc9bcd207e915d7924b HTTP/1.1" 200 None 2026-03-10 06:57:35,337 - lava_callback - ERROR - Error processing callback for node 69ad1dc9bcd207e915d7924b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55766 - "GET / HTTP/1.0" 200 OK 2026-03-10 06:57:55,264 - lava_callback - INFO - Received callback for node 69ad1defbcd207e915d79267 INFO: 172.20.0.1:33638 - "POST /node/69ad1defbcd207e915d79267 HTTP/1.0" 202 Accepted 2026-03-10 06:57:55,271 - lava_callback - INFO - Starting async job submit for node 69ad1defbcd207e915d79267 2026-03-10 06:57:55,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:57:55,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad1defbcd207e915d79267 HTTP/1.1" 200 None 2026-03-10 06:57:55,318 - lava_callback - DEBUG - Processing node 69ad1defbcd207e915d79267: baseline-arm-clabbe 2026-03-10 06:57:55,318 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ad1defbcd207e915d79267 2026-03-10 06:57:55,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:57:55,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad1defbcd207e915d79267 HTTP/1.1" 200 None 2026-03-10 06:57:55,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:57:55,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acc009bcd207e915d76b94 HTTP/1.1" 200 None 2026-03-10 06:57:55,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:57:55,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ad1defbcd207e915d79267 HTTP/1.1" 200 None 2026-03-10 06:57:55,447 - lava_callback - ERROR - Error processing callback for node 69ad1defbcd207e915d79267: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44622 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37112 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36506 - "GET / HTTP/1.0" 200 OK 2026-03-10 06:59:30,952 - lava_callback - INFO - Received callback for node 69ad1e51bcd207e915d79290 INFO: 172.20.0.1:58358 - "POST /node/69ad1e51bcd207e915d79290 HTTP/1.0" 202 Accepted 2026-03-10 06:59:30,957 - lava_callback - INFO - Starting async job submit for node 69ad1e51bcd207e915d79290 2026-03-10 06:59:30,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:59:30,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad1e51bcd207e915d79290 HTTP/1.1" 200 None 2026-03-10 06:59:30,973 - lava_callback - DEBUG - Processing node 69ad1e51bcd207e915d79290: baseline-arm64-baylibre 2026-03-10 06:59:30,974 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ad1e51bcd207e915d79290 2026-03-10 06:59:30,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:59:30,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad1e51bcd207e915d79290 HTTP/1.1" 200 None 2026-03-10 06:59:30,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:59:31,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa1d4a8c4d487a0e751161 HTTP/1.1" 200 None 2026-03-10 06:59:31,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 06:59:31,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ad1e51bcd207e915d79290 HTTP/1.1" 200 None 2026-03-10 06:59:31,037 - lava_callback - ERROR - Error processing callback for node 69ad1e51bcd207e915d79290: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58042 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56984 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38958 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38422 - "GET / HTTP/1.0" 200 OK 2026-03-10 07:02:51,039 - lava_callback - INFO - Received callback for node 69ad1f07bcd207e915d792b9 INFO: 172.20.0.1:46648 - "POST /node/69ad1f07bcd207e915d792b9 HTTP/1.0" 202 Accepted 2026-03-10 07:02:51,045 - lava_callback - INFO - Starting async job submit for node 69ad1f07bcd207e915d792b9 2026-03-10 07:02:51,045 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:02:51,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad1f07bcd207e915d792b9 HTTP/1.1" 200 None 2026-03-10 07:02:51,060 - lava_callback - DEBUG - Processing node 69ad1f07bcd207e915d792b9: baseline-arm64-baylibre 2026-03-10 07:02:51,060 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ad1f07bcd207e915d792b9 2026-03-10 07:02:51,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:02:51,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad1f07bcd207e915d792b9 HTTP/1.1" 200 None 2026-03-10 07:02:51,076 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:02:51,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa1d4a8c4d487a0e751161 HTTP/1.1" 200 None 2026-03-10 07:02:51,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:02:51,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ad1f07bcd207e915d792b9 HTTP/1.1" 200 None 2026-03-10 07:02:51,120 - lava_callback - ERROR - Error processing callback for node 69ad1f07bcd207e915d792b9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57100 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53764 - "GET / HTTP/1.0" 200 OK 2026-03-10 07:04:01,093 - lava_callback - INFO - Received callback for node 69af9b14275c8d3b7939b038 INFO: 172.20.0.1:51506 - "POST /node/69af9b14275c8d3b7939b038 HTTP/1.0" 202 Accepted 2026-03-10 07:04:01,099 - lava_callback - INFO - Starting async job submit for node 69af9b14275c8d3b7939b038 2026-03-10 07:04:01,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:04:01,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af9b14275c8d3b7939b038 HTTP/1.1" 200 None 2026-03-10 07:04:01,168 - lava_callback - DEBUG - Processing node 69af9b14275c8d3b7939b038: baseline-arm64-broonie 2026-03-10 07:04:01,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 07:04:01,697 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 07:04:01,699 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af9b14275c8d3b7939b038/log.txt.gz 2026-03-10 07:04:01,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 07:04:02,164 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 07:04:02,166 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af9b14275c8d3b7939b038/lava-logs.yaml 2026-03-10 07:04:02,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 07:04:02,325 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 07:04:02,328 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af9b14275c8d3b7939b038/lava_callback.json.gz DEBUG: Setup failed for 69af9b14275c8d3b7939b038. Transitting job node result: incomplete -> fail 2026-03-10 07:04:02,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:04:02,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af9b14275c8d3b7939b038 HTTP/1.1" 200 None 2026-03-10 07:04:02,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:04:02,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aeba41feb96b28aa34a78b HTTP/1.1" 200 None 2026-03-10 07:04:02,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:04:02,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af9b14275c8d3b7939b038 HTTP/1.1" 200 None 2026-03-10 07:04:02,527 - lava_callback - INFO - Completed processing callback for node 69af9b14275c8d3b7939b038 INFO: 172.20.0.1:51514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52634 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-10 07:04:15,580 - lava_callback - INFO - Received callback for node 69ad1f59bcd207e915d792d7 INFO: 172.20.0.1:52644 - "POST /node/69ad1f59bcd207e915d792d7 HTTP/1.0" 202 Accepted 2026-03-10 07:04:15,586 - lava_callback - INFO - Starting async job submit for node 69ad1f59bcd207e915d792d7 2026-03-10 07:04:15,586 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:04:15,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad1f59bcd207e915d792d7 HTTP/1.1" 200 None 2026-03-10 07:04:15,611 - lava_callback - DEBUG - Processing node 69ad1f59bcd207e915d792d7: baseline-arm64-clabbe 2026-03-10 07:04:15,612 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ad1f59bcd207e915d792d7 2026-03-10 07:04:15,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:04:15,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad1f59bcd207e915d792d7 HTTP/1.1" 200 None 2026-03-10 07:04:15,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:04:15,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa1d4a8c4d487a0e751161 HTTP/1.1" 200 None 2026-03-10 07:04:15,669 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:04:15,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ad1f59bcd207e915d792d7 HTTP/1.1" 200 None 2026-03-10 07:04:15,691 - lava_callback - ERROR - Error processing callback for node 69ad1f59bcd207e915d792d7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52650 - "GET / HTTP/1.0" 200 OK 2026-03-10 07:04:24,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:04:24,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:43578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40516 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-10 07:05:15,613 - lava_callback - INFO - Received callback for node 69ad1fa4bcd207e915d792eb INFO: 172.20.0.1:40528 - "POST /node/69ad1fa4bcd207e915d792eb HTTP/1.0" 202 Accepted 2026-03-10 07:05:15,618 - lava_callback - INFO - Starting async job submit for node 69ad1fa4bcd207e915d792eb 2026-03-10 07:05:15,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:05:15,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad1fa4bcd207e915d792eb HTTP/1.1" 200 None 2026-03-10 07:05:15,636 - lava_callback - DEBUG - Processing node 69ad1fa4bcd207e915d792eb: baseline-arm64-clabbe 2026-03-10 07:05:15,636 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ad1fa4bcd207e915d792eb 2026-03-10 07:05:15,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:05:15,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad1fa4bcd207e915d792eb HTTP/1.1" 200 None 2026-03-10 07:05:15,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:05:15,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa1d4a8c4d487a0e751161 HTTP/1.1" 200 None 2026-03-10 07:05:15,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:05:15,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ad1fa4bcd207e915d792eb HTTP/1.1" 200 None 2026-03-10 07:05:15,700 - lava_callback - ERROR - Error processing callback for node 69ad1fa4bcd207e915d792eb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40764 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40392 - "GET / HTTP/1.0" 200 OK 2026-03-10 07:06:35,650 - lava_callback - INFO - Received callback for node 69ad1fefbcd207e915d792fd INFO: 172.20.0.1:36452 - "POST /node/69ad1fefbcd207e915d792fd HTTP/1.0" 202 Accepted 2026-03-10 07:06:35,656 - lava_callback - INFO - Starting async job submit for node 69ad1fefbcd207e915d792fd 2026-03-10 07:06:35,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:06:35,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad1fefbcd207e915d792fd HTTP/1.1" 200 None 2026-03-10 07:06:35,672 - lava_callback - DEBUG - Processing node 69ad1fefbcd207e915d792fd: baseline-arm64-clabbe 2026-03-10 07:06:35,673 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ad1fefbcd207e915d792fd 2026-03-10 07:06:35,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:06:35,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad1fefbcd207e915d792fd HTTP/1.1" 200 None 2026-03-10 07:06:35,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:06:35,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa1d4a8c4d487a0e751161 HTTP/1.1" 200 None 2026-03-10 07:06:35,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:06:35,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ad1fefbcd207e915d792fd HTTP/1.1" 200 None 2026-03-10 07:06:35,729 - lava_callback - ERROR - Error processing callback for node 69ad1fefbcd207e915d792fd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36454 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33712 - "GET / HTTP/1.0" 200 OK 2026-03-10 07:07:55,728 - lava_callback - INFO - Received callback for node 69ad203cbcd207e915d7930c INFO: 172.20.0.1:37812 - "POST /node/69ad203cbcd207e915d7930c HTTP/1.0" 202 Accepted 2026-03-10 07:07:55,734 - lava_callback - INFO - Starting async job submit for node 69ad203cbcd207e915d7930c 2026-03-10 07:07:55,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:07:55,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad203cbcd207e915d7930c HTTP/1.1" 200 None 2026-03-10 07:07:55,754 - lava_callback - DEBUG - Processing node 69ad203cbcd207e915d7930c: baseline-arm64-clabbe 2026-03-10 07:07:55,754 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ad203cbcd207e915d7930c 2026-03-10 07:07:55,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:07:55,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad203cbcd207e915d7930c HTTP/1.1" 200 None 2026-03-10 07:07:55,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:07:55,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa1d4a8c4d487a0e751161 HTTP/1.1" 200 None 2026-03-10 07:07:55,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:07:55,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ad203cbcd207e915d7930c HTTP/1.1" 200 None 2026-03-10 07:07:55,826 - lava_callback - ERROR - Error processing callback for node 69ad203cbcd207e915d7930c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59066 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48978 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58718 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33392 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51360 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36504 - "GET / HTTP/1.0" 200 OK 2026-03-10 07:13:01,457 - lava_callback - INFO - Received callback for node 69afafae275c8d3b7939b409 INFO: 172.20.0.1:54142 - "POST /node/69afafae275c8d3b7939b409 HTTP/1.0" 202 Accepted 2026-03-10 07:13:01,464 - lava_callback - INFO - Starting async job submit for node 69afafae275c8d3b7939b409 2026-03-10 07:13:01,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:13:01,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afafae275c8d3b7939b409 HTTP/1.1" 200 None 2026-03-10 07:13:01,540 - lava_callback - DEBUG - Processing node 69afafae275c8d3b7939b409: baseline-arm64-broonie 2026-03-10 07:13:01,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 07:13:02,092 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 07:13:02,094 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69afafae275c8d3b7939b409/log.txt.gz 2026-03-10 07:13:02,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 07:13:02,536 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 07:13:02,537 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69afafae275c8d3b7939b409/lava-logs.yaml 2026-03-10 07:13:02,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 07:13:02,826 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 07:13:02,828 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69afafae275c8d3b7939b409/lava_callback.json.gz DEBUG: Setup failed for 69afafae275c8d3b7939b409. Transitting job node result: incomplete -> fail 2026-03-10 07:13:02,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:13:02,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afafae275c8d3b7939b409 HTTP/1.1" 200 None 2026-03-10 07:13:02,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:13:02,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-10 07:13:02,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:13:02,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afafae275c8d3b7939b409 HTTP/1.1" 200 None 2026-03-10 07:13:03,035 - lava_callback - INFO - Completed processing callback for node 69afafae275c8d3b7939b409 INFO: 172.20.0.1:54154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50834 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50836 - "GET / HTTP/1.0" 200 OK 2026-03-10 07:13:24,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:13:24,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:45638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60508 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43466 - "GET / HTTP/1.0" 200 OK 2026-03-10 07:14:31,928 - lava_callback - INFO - Received callback for node 69ad21d2bcd207e915d79350 INFO: 172.20.0.1:58430 - "POST /node/69ad21d2bcd207e915d79350 HTTP/1.0" 202 Accepted 2026-03-10 07:14:31,933 - lava_callback - INFO - Starting async job submit for node 69ad21d2bcd207e915d79350 2026-03-10 07:14:31,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:14:31,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad21d2bcd207e915d79350 HTTP/1.1" 200 None 2026-03-10 07:14:31,960 - lava_callback - DEBUG - Processing node 69ad21d2bcd207e915d79350: baseline-arm64-baylibre 2026-03-10 07:14:31,960 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ad21d2bcd207e915d79350 2026-03-10 07:14:31,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:14:31,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad21d2bcd207e915d79350 HTTP/1.1" 200 None 2026-03-10 07:14:31,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:14:31,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a77a6ba255cc0324eb011a HTTP/1.1" 200 None 2026-03-10 07:14:31,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:14:32,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ad21d2bcd207e915d79350 HTTP/1.1" 200 None 2026-03-10 07:14:32,023 - lava_callback - ERROR - Error processing callback for node 69ad21d2bcd207e915d79350: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55116 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38612 - "GET / HTTP/1.0" 200 OK 2026-03-10 07:15:51,724 - lava_callback - INFO - Received callback for node 69ad2222bcd207e915d79366 INFO: 172.20.0.1:59638 - "POST /node/69ad2222bcd207e915d79366 HTTP/1.0" 202 Accepted 2026-03-10 07:15:51,729 - lava_callback - INFO - Starting async job submit for node 69ad2222bcd207e915d79366 2026-03-10 07:15:51,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:15:51,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad2222bcd207e915d79366 HTTP/1.1" 200 None 2026-03-10 07:15:51,745 - lava_callback - DEBUG - Processing node 69ad2222bcd207e915d79366: baseline-arm64-baylibre 2026-03-10 07:15:51,745 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ad2222bcd207e915d79366 2026-03-10 07:15:51,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:15:51,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad2222bcd207e915d79366 HTTP/1.1" 200 None 2026-03-10 07:15:51,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:15:51,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a77a6ba255cc0324eb011a HTTP/1.1" 200 None 2026-03-10 07:15:51,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:15:51,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ad2222bcd207e915d79366 HTTP/1.1" 200 None 2026-03-10 07:15:51,817 - lava_callback - ERROR - Error processing callback for node 69ad2222bcd207e915d79366: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53570 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54562 - "GET / HTTP/1.0" 200 OK 2026-03-10 07:17:12,066 - lava_callback - INFO - Received callback for node 69ad2273bcd207e915d79376 INFO: 172.20.0.1:42444 - "POST /node/69ad2273bcd207e915d79376 HTTP/1.0" 202 Accepted 2026-03-10 07:17:12,071 - lava_callback - INFO - Starting async job submit for node 69ad2273bcd207e915d79376 2026-03-10 07:17:12,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:17:12,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad2273bcd207e915d79376 HTTP/1.1" 200 None 2026-03-10 07:17:12,096 - lava_callback - DEBUG - Processing node 69ad2273bcd207e915d79376: baseline-arm64-baylibre 2026-03-10 07:17:12,097 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ad2273bcd207e915d79376 2026-03-10 07:17:12,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:17:12,110 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad2273bcd207e915d79376 HTTP/1.1" 200 None 2026-03-10 07:17:12,112 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:17:12,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a77a6ba255cc0324eb011a HTTP/1.1" 200 None 2026-03-10 07:17:12,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:17:12,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ad2273bcd207e915d79376 HTTP/1.1" 200 None 2026-03-10 07:17:12,154 - lava_callback - ERROR - Error processing callback for node 69ad2273bcd207e915d79376: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42452 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34894 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59082 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51996 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42178 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49970 - "GET / HTTP/1.0" 200 OK 2026-03-10 07:22:01,767 - lava_callback - INFO - Received callback for node 69afbe3a275c8d3b7939b573 INFO: 172.20.0.1:46700 - "POST /node/69afbe3a275c8d3b7939b573 HTTP/1.0" 202 Accepted 2026-03-10 07:22:01,774 - lava_callback - INFO - Starting async job submit for node 69afbe3a275c8d3b7939b573 2026-03-10 07:22:01,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:22:01,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afbe3a275c8d3b7939b573 HTTP/1.1" 200 None 2026-03-10 07:22:01,825 - lava_callback - DEBUG - Processing node 69afbe3a275c8d3b7939b573: baseline-arm64-broonie 2026-03-10 07:22:02,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 07:22:02,321 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 07:22:02,322 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69afbe3a275c8d3b7939b573/log.txt.gz 2026-03-10 07:22:02,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 07:22:02,792 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 07:22:02,794 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69afbe3a275c8d3b7939b573/lava-logs.yaml 2026-03-10 07:22:02,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 07:22:02,939 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 07:22:02,941 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69afbe3a275c8d3b7939b573/lava_callback.json.gz DEBUG: Setup failed for 69afbe3a275c8d3b7939b573. Transitting job node result: incomplete -> fail 2026-03-10 07:22:03,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:22:03,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afbe3a275c8d3b7939b573 HTTP/1.1" 200 None 2026-03-10 07:22:03,030 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:22:03,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aeba41feb96b28aa34a78b HTTP/1.1" 200 None 2026-03-10 07:22:03,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:22:03,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afbe3a275c8d3b7939b573 HTTP/1.1" 200 None 2026-03-10 07:22:03,155 - lava_callback - INFO - Completed processing callback for node 69afbe3a275c8d3b7939b573 INFO: 172.20.0.1:46702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34780 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34788 - "GET / HTTP/1.0" 200 OK 2026-03-10 07:22:24,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:22:24,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:52838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41402 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42008 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45814 - "GET / HTTP/1.0" 200 OK 2026-03-10 07:24:32,239 - lava_callback - INFO - Received callback for node 69ad242cbcd207e915d79412 INFO: 172.20.0.1:55288 - "POST /node/69ad242cbcd207e915d79412 HTTP/1.0" 202 Accepted 2026-03-10 07:24:32,245 - lava_callback - INFO - Starting async job submit for node 69ad242cbcd207e915d79412 2026-03-10 07:24:32,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:24:32,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad242cbcd207e915d79412 HTTP/1.1" 200 None 2026-03-10 07:24:32,274 - lava_callback - DEBUG - Processing node 69ad242cbcd207e915d79412: baseline-arm64-baylibre 2026-03-10 07:24:32,274 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ad242cbcd207e915d79412 2026-03-10 07:24:32,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:24:32,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad242cbcd207e915d79412 HTTP/1.1" 200 None 2026-03-10 07:24:32,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:24:32,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a77a6ba255cc0324eb011a HTTP/1.1" 200 None 2026-03-10 07:24:32,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:24:32,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ad242cbcd207e915d79412 HTTP/1.1" 200 None 2026-03-10 07:24:32,340 - lava_callback - ERROR - Error processing callback for node 69ad242cbcd207e915d79412: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57214 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44822 - "GET / HTTP/1.0" 200 OK 2026-03-10 07:25:52,319 - lava_callback - INFO - Received callback for node 69ad2476bcd207e915d7941e INFO: 172.20.0.1:44066 - "POST /node/69ad2476bcd207e915d7941e HTTP/1.0" 202 Accepted 2026-03-10 07:25:52,324 - lava_callback - INFO - Starting async job submit for node 69ad2476bcd207e915d7941e 2026-03-10 07:25:52,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:25:52,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad2476bcd207e915d7941e HTTP/1.1" 200 None 2026-03-10 07:25:52,340 - lava_callback - DEBUG - Processing node 69ad2476bcd207e915d7941e: baseline-arm64-baylibre 2026-03-10 07:25:52,341 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ad2476bcd207e915d7941e 2026-03-10 07:25:52,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:25:52,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad2476bcd207e915d7941e HTTP/1.1" 200 None 2026-03-10 07:25:52,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:25:52,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acc01dbcd207e915d76ba7 HTTP/1.1" 200 None 2026-03-10 07:25:52,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:25:52,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ad2476bcd207e915d7941e HTTP/1.1" 200 None 2026-03-10 07:25:52,393 - lava_callback - ERROR - Error processing callback for node 69ad2476bcd207e915d7941e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37706 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59206 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37596 - "GET / HTTP/1.0" 200 OK 2026-03-10 07:27:32,360 - lava_callback - INFO - Received callback for node 69ad24e1bcd207e915d7944f INFO: 172.20.0.1:46460 - "POST /node/69ad24e1bcd207e915d7944f HTTP/1.0" 202 Accepted 2026-03-10 07:27:32,365 - lava_callback - INFO - Starting async job submit for node 69ad24e1bcd207e915d7944f 2026-03-10 07:27:32,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:27:32,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad24e1bcd207e915d7944f HTTP/1.1" 200 None 2026-03-10 07:27:32,382 - lava_callback - DEBUG - Processing node 69ad24e1bcd207e915d7944f: baseline-arm64-baylibre 2026-03-10 07:27:32,383 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ad24e1bcd207e915d7944f 2026-03-10 07:27:32,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:27:32,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad24e1bcd207e915d7944f HTTP/1.1" 200 None 2026-03-10 07:27:32,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:27:32,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acc01dbcd207e915d76ba7 HTTP/1.1" 200 None 2026-03-10 07:27:32,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:27:32,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ad24e1bcd207e915d7944f HTTP/1.1" 200 None 2026-03-10 07:27:32,475 - lava_callback - ERROR - Error processing callback for node 69ad24e1bcd207e915d7944f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42744 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40458 - "GET / HTTP/1.0" 200 OK 2026-03-10 07:28:52,467 - lava_callback - INFO - Received callback for node 69ad252cbcd207e915d7945f INFO: 172.20.0.1:52928 - "POST /node/69ad252cbcd207e915d7945f HTTP/1.0" 202 Accepted 2026-03-10 07:28:52,472 - lava_callback - INFO - Starting async job submit for node 69ad252cbcd207e915d7945f 2026-03-10 07:28:52,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:28:52,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad252cbcd207e915d7945f HTTP/1.1" 200 None 2026-03-10 07:28:52,488 - lava_callback - DEBUG - Processing node 69ad252cbcd207e915d7945f: baseline-arm64-baylibre 2026-03-10 07:28:52,488 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ad252cbcd207e915d7945f 2026-03-10 07:28:52,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:28:52,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad252cbcd207e915d7945f HTTP/1.1" 200 None 2026-03-10 07:28:52,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:28:52,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acc01dbcd207e915d76ba7 HTTP/1.1" 200 None 2026-03-10 07:28:52,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:28:52,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ad252cbcd207e915d7945f HTTP/1.1" 200 None 2026-03-10 07:28:52,560 - lava_callback - ERROR - Error processing callback for node 69ad252cbcd207e915d7945f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37326 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59670 - "GET / HTTP/1.0" 200 OK 2026-03-10 07:30:12,646 - lava_callback - INFO - Received callback for node 69ad2576bcd207e915d7947d INFO: 172.20.0.1:40508 - "POST /node/69ad2576bcd207e915d7947d HTTP/1.0" 202 Accepted 2026-03-10 07:30:12,651 - lava_callback - INFO - Starting async job submit for node 69ad2576bcd207e915d7947d 2026-03-10 07:30:12,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:30:12,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad2576bcd207e915d7947d HTTP/1.1" 200 None 2026-03-10 07:30:12,668 - lava_callback - DEBUG - Processing node 69ad2576bcd207e915d7947d: baseline-arm64-baylibre 2026-03-10 07:30:12,669 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ad2576bcd207e915d7947d 2026-03-10 07:30:12,669 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:30:12,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad2576bcd207e915d7947d HTTP/1.1" 200 None 2026-03-10 07:30:12,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:30:12,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acc01dbcd207e915d76ba7 HTTP/1.1" 200 None 2026-03-10 07:30:12,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:30:12,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ad2576bcd207e915d7947d HTTP/1.1" 200 None 2026-03-10 07:30:12,728 - lava_callback - ERROR - Error processing callback for node 69ad2576bcd207e915d7947d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40510 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54164 - "GET / HTTP/1.0" 200 OK 2026-03-10 07:31:02,148 - lava_callback - INFO - Received callback for node 69afbfac275c8d3b7939b5d4 INFO: 172.20.0.1:34622 - "POST /node/69afbfac275c8d3b7939b5d4 HTTP/1.0" 202 Accepted 2026-03-10 07:31:02,154 - lava_callback - INFO - Starting async job submit for node 69afbfac275c8d3b7939b5d4 2026-03-10 07:31:02,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:31:02,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afbfac275c8d3b7939b5d4 HTTP/1.1" 200 None 2026-03-10 07:31:02,203 - lava_callback - DEBUG - Processing node 69afbfac275c8d3b7939b5d4: baseline-arm64-broonie 2026-03-10 07:31:02,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 07:31:02,688 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 07:31:02,690 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69afbfac275c8d3b7939b5d4/log.txt.gz 2026-03-10 07:31:02,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 07:31:03,128 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 07:31:03,130 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69afbfac275c8d3b7939b5d4/lava-logs.yaml 2026-03-10 07:31:03,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 07:31:03,296 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 07:31:03,298 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69afbfac275c8d3b7939b5d4/lava_callback.json.gz DEBUG: Setup failed for 69afbfac275c8d3b7939b5d4. Transitting job node result: incomplete -> fail 2026-03-10 07:31:03,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:31:03,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afbfac275c8d3b7939b5d4 HTTP/1.1" 200 None 2026-03-10 07:31:03,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:31:03,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-10 07:31:03,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:31:03,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afbfac275c8d3b7939b5d4 HTTP/1.1" 200 None 2026-03-10 07:31:03,505 - lava_callback - INFO - Completed processing callback for node 69afbfac275c8d3b7939b5d4 INFO: 172.20.0.1:34638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35664 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35676 - "GET / HTTP/1.0" 200 OK 2026-03-10 07:31:24,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:31:25,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:36068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37992 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40454 - "GET / HTTP/1.0" 200 OK 2026-03-10 07:32:36,925 - lava_callback - INFO - Received callback for node 69ad2610bcd207e915d794ad INFO: 172.20.0.1:40458 - "POST /node/69ad2610bcd207e915d794ad HTTP/1.0" 202 Accepted 2026-03-10 07:32:36,931 - lava_callback - INFO - Starting async job submit for node 69ad2610bcd207e915d794ad 2026-03-10 07:32:36,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:32:36,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad2610bcd207e915d794ad HTTP/1.1" 200 None 2026-03-10 07:32:36,947 - lava_callback - DEBUG - Processing node 69ad2610bcd207e915d794ad: baseline-arm64-clabbe 2026-03-10 07:32:36,947 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ad2610bcd207e915d794ad 2026-03-10 07:32:36,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:32:36,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad2610bcd207e915d794ad HTTP/1.1" 200 None 2026-03-10 07:32:36,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:32:36,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acc01dbcd207e915d76ba7 HTTP/1.1" 200 None 2026-03-10 07:32:36,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:32:36,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ad2610bcd207e915d794ad HTTP/1.1" 200 None 2026-03-10 07:32:37,006 - lava_callback - ERROR - Error processing callback for node 69ad2610bcd207e915d794ad: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40520 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52210 - "GET / HTTP/1.0" 200 OK 2026-03-10 07:33:56,944 - lava_callback - INFO - Received callback for node 69ad265cbcd207e915d794c3 INFO: 172.20.0.1:52216 - "POST /node/69ad265cbcd207e915d794c3 HTTP/1.0" 202 Accepted 2026-03-10 07:33:56,949 - lava_callback - INFO - Starting async job submit for node 69ad265cbcd207e915d794c3 2026-03-10 07:33:56,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:33:56,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad265cbcd207e915d794c3 HTTP/1.1" 200 None 2026-03-10 07:33:56,966 - lava_callback - DEBUG - Processing node 69ad265cbcd207e915d794c3: baseline-arm64-clabbe 2026-03-10 07:33:56,966 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ad265cbcd207e915d794c3 2026-03-10 07:33:56,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:33:56,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad265cbcd207e915d794c3 HTTP/1.1" 200 None 2026-03-10 07:33:56,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:33:56,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acc01dbcd207e915d76ba7 HTTP/1.1" 200 None 2026-03-10 07:33:56,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:33:57,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ad265cbcd207e915d794c3 HTTP/1.1" 200 None 2026-03-10 07:33:57,027 - lava_callback - ERROR - Error processing callback for node 69ad265cbcd207e915d794c3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39606 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39946 - "GET / HTTP/1.0" 200 OK 2026-03-10 07:35:13,012 - lava_callback - INFO - Received callback for node 69ad26a1bcd207e915d794db INFO: 172.20.0.1:54574 - "POST /node/69ad26a1bcd207e915d794db HTTP/1.0" 202 Accepted 2026-03-10 07:35:13,017 - lava_callback - INFO - Starting async job submit for node 69ad26a1bcd207e915d794db 2026-03-10 07:35:13,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:35:13,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad26a1bcd207e915d794db HTTP/1.1" 200 None 2026-03-10 07:35:13,034 - lava_callback - DEBUG - Processing node 69ad26a1bcd207e915d794db: baseline-arm64-baylibre 2026-03-10 07:35:13,034 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ad26a1bcd207e915d794db 2026-03-10 07:35:13,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:35:13,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad26a1bcd207e915d794db HTTP/1.1" 200 None 2026-03-10 07:35:13,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:35:13,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a77a6ba255cc0324eb011a HTTP/1.1" 200 None 2026-03-10 07:35:13,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:35:13,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ad26a1bcd207e915d794db HTTP/1.1" 200 None 2026-03-10 07:35:13,116 - lava_callback - ERROR - Error processing callback for node 69ad26a1bcd207e915d794db: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54578 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34302 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34118 - "GET / HTTP/1.0" 200 OK 2026-03-10 07:36:32,791 - lava_callback - INFO - Received callback for node 69ad26ebbcd207e915d794fc INFO: 172.20.0.1:48396 - "POST /node/69ad26ebbcd207e915d794fc HTTP/1.0" 202 Accepted 2026-03-10 07:36:32,796 - lava_callback - INFO - Starting async job submit for node 69ad26ebbcd207e915d794fc 2026-03-10 07:36:32,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:36:32,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad26ebbcd207e915d794fc HTTP/1.1" 200 None 2026-03-10 07:36:32,817 - lava_callback - DEBUG - Processing node 69ad26ebbcd207e915d794fc: baseline-arm64-baylibre 2026-03-10 07:36:32,817 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ad26ebbcd207e915d794fc 2026-03-10 07:36:32,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:36:32,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad26ebbcd207e915d794fc HTTP/1.1" 200 None 2026-03-10 07:36:32,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:36:32,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a77a6ba255cc0324eb011a HTTP/1.1" 200 None 2026-03-10 07:36:32,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:36:32,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ad26ebbcd207e915d794fc HTTP/1.1" 200 None 2026-03-10 07:36:32,877 - lava_callback - ERROR - Error processing callback for node 69ad26ebbcd207e915d794fc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38528 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43692 - "GET / HTTP/1.0" 200 OK 2026-03-10 07:37:32,800 - lava_callback - INFO - Received callback for node 69ad2734bcd207e915d79514 INFO: 172.20.0.1:60154 - "POST /node/69ad2734bcd207e915d79514 HTTP/1.0" 202 Accepted 2026-03-10 07:37:32,806 - lava_callback - INFO - Starting async job submit for node 69ad2734bcd207e915d79514 2026-03-10 07:37:32,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:37:32,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad2734bcd207e915d79514 HTTP/1.1" 200 None 2026-03-10 07:37:32,822 - lava_callback - DEBUG - Processing node 69ad2734bcd207e915d79514: baseline-arm64-baylibre 2026-03-10 07:37:32,822 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ad2734bcd207e915d79514 2026-03-10 07:37:32,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:37:32,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad2734bcd207e915d79514 HTTP/1.1" 200 None 2026-03-10 07:37:32,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:37:32,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a77a6ba255cc0324eb011a HTTP/1.1" 200 None 2026-03-10 07:37:32,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:37:32,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ad2734bcd207e915d79514 HTTP/1.1" 200 None 2026-03-10 07:37:32,890 - lava_callback - ERROR - Error processing callback for node 69ad2734bcd207e915d79514: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59824 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35152 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48372 - "GET / HTTP/1.0" 200 OK 2026-03-10 07:40:13,021 - lava_callback - INFO - Received callback for node 69ad27cabcd207e915d7953c INFO: 172.20.0.1:58632 - "POST /node/69ad27cabcd207e915d7953c HTTP/1.0" 202 Accepted 2026-03-10 07:40:13,026 - lava_callback - INFO - Starting async job submit for node 69ad27cabcd207e915d7953c 2026-03-10 07:40:13,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:40:13,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad27cabcd207e915d7953c HTTP/1.1" 200 None 2026-03-10 07:40:13,092 - lava_callback - DEBUG - Processing node 69ad27cabcd207e915d7953c: baseline-arm64-baylibre 2026-03-10 07:40:13,093 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ad27cabcd207e915d7953c 2026-03-10 07:40:13,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:40:13,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad27cabcd207e915d7953c HTTP/1.1" 200 None 2026-03-10 07:40:13,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:40:13,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a77a6ba255cc0324eb011a HTTP/1.1" 200 None 2026-03-10 07:40:13,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:40:13,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ad27cabcd207e915d7953c HTTP/1.1" 200 None 2026-03-10 07:40:13,171 - lava_callback - ERROR - Error processing callback for node 69ad27cabcd207e915d7953c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58642 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53880 - "GET / HTTP/1.0" 200 OK 2026-03-10 07:40:42,464 - lava_callback - INFO - Received callback for node 69afc0e6275c8d3b7939b62e INFO: 172.20.0.1:36614 - "POST /node/69afc0e6275c8d3b7939b62e HTTP/1.0" 202 Accepted 2026-03-10 07:40:42,470 - lava_callback - INFO - Starting async job submit for node 69afc0e6275c8d3b7939b62e 2026-03-10 07:40:42,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:40:42,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afc0e6275c8d3b7939b62e HTTP/1.1" 200 None 2026-03-10 07:40:42,523 - lava_callback - DEBUG - Processing node 69afc0e6275c8d3b7939b62e: baseline-arm64-broonie 2026-03-10 07:40:42,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 07:40:43,034 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 07:40:43,036 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69afc0e6275c8d3b7939b62e/log.txt.gz 2026-03-10 07:40:43,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 07:40:43,496 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 07:40:43,498 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69afc0e6275c8d3b7939b62e/lava-logs.yaml 2026-03-10 07:40:43,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 07:40:43,656 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 07:40:43,658 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69afc0e6275c8d3b7939b62e/lava_callback.json.gz DEBUG: Setup failed for 69afc0e6275c8d3b7939b62e. Transitting job node result: incomplete -> fail 2026-03-10 07:40:43,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:40:43,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afc0e6275c8d3b7939b62e HTTP/1.1" 200 None 2026-03-10 07:40:43,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:40:43,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-10 07:40:43,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:40:43,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afc0e6275c8d3b7939b62e HTTP/1.1" 200 None 2026-03-10 07:40:43,867 - lava_callback - INFO - Completed processing callback for node 69afc0e6275c8d3b7939b62e INFO: 172.20.0.1:36624 - "GET / HTTP/1.0" 200 OK 2026-03-10 07:40:55,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:40:55,044 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:34514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39708 - "GET / HTTP/1.0" 200 OK 2026-03-10 07:41:13,020 - lava_callback - INFO - Received callback for node 69ad2817bcd207e915d79556 INFO: 172.20.0.1:55742 - "POST /node/69ad2817bcd207e915d79556 HTTP/1.0" 202 Accepted 2026-03-10 07:41:13,026 - lava_callback - INFO - Starting async job submit for node 69ad2817bcd207e915d79556 2026-03-10 07:41:13,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:41:13,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad2817bcd207e915d79556 HTTP/1.1" 200 None 2026-03-10 07:41:13,042 - lava_callback - DEBUG - Processing node 69ad2817bcd207e915d79556: baseline-arm64-baylibre 2026-03-10 07:41:13,042 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ad2817bcd207e915d79556 2026-03-10 07:41:13,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:41:13,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad2817bcd207e915d79556 HTTP/1.1" 200 None 2026-03-10 07:41:13,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:41:13,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a77a6ba255cc0324eb011a HTTP/1.1" 200 None 2026-03-10 07:41:13,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:41:13,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ad2817bcd207e915d79556 HTTP/1.1" 200 None 2026-03-10 07:41:13,106 - lava_callback - ERROR - Error processing callback for node 69ad2817bcd207e915d79556: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55746 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39772 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38212 - "GET / HTTP/1.0" 200 OK 2026-03-10 07:42:33,097 - lava_callback - INFO - Received callback for node 69ad2862bcd207e915d79579 INFO: 172.20.0.1:50476 - "POST /node/69ad2862bcd207e915d79579 HTTP/1.0" 202 Accepted 2026-03-10 07:42:33,103 - lava_callback - INFO - Starting async job submit for node 69ad2862bcd207e915d79579 2026-03-10 07:42:33,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:42:33,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad2862bcd207e915d79579 HTTP/1.1" 200 None 2026-03-10 07:42:33,140 - lava_callback - DEBUG - Processing node 69ad2862bcd207e915d79579: baseline-arm64-baylibre 2026-03-10 07:42:33,140 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ad2862bcd207e915d79579 2026-03-10 07:42:33,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:42:33,154 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad2862bcd207e915d79579 HTTP/1.1" 200 None 2026-03-10 07:42:33,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:42:33,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a77a6ba255cc0324eb011a HTTP/1.1" 200 None 2026-03-10 07:42:33,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:42:33,201 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ad2862bcd207e915d79579 HTTP/1.1" 200 None 2026-03-10 07:42:33,202 - lava_callback - ERROR - Error processing callback for node 69ad2862bcd207e915d79579: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35648 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53726 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55686 - "GET / HTTP/1.0" 200 OK 2026-03-10 07:44:53,297 - lava_callback - INFO - Received callback for node 69ad28e2bcd207e915d795a1 INFO: 172.20.0.1:59658 - "POST /node/69ad28e2bcd207e915d795a1 HTTP/1.0" 202 Accepted 2026-03-10 07:44:53,303 - lava_callback - INFO - Starting async job submit for node 69ad28e2bcd207e915d795a1 2026-03-10 07:44:53,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:44:53,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad28e2bcd207e915d795a1 HTTP/1.1" 200 None 2026-03-10 07:44:53,320 - lava_callback - DEBUG - Processing node 69ad28e2bcd207e915d795a1: baseline-arm64-baylibre 2026-03-10 07:44:53,320 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ad28e2bcd207e915d795a1 2026-03-10 07:44:53,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:44:53,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad28e2bcd207e915d795a1 HTTP/1.1" 200 None 2026-03-10 07:44:53,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:44:53,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a77a6ba255cc0324eb011a HTTP/1.1" 200 None 2026-03-10 07:44:53,376 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:44:53,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ad28e2bcd207e915d795a1 HTTP/1.1" 200 None 2026-03-10 07:44:53,433 - lava_callback - ERROR - Error processing callback for node 69ad28e2bcd207e915d795a1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40688 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43386 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59630 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50750 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40292 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43002 - "GET / HTTP/1.0" 200 OK 2026-03-10 07:50:03,101 - lava_callback - INFO - Received callback for node 69afc649275c8d3b7939b706 INFO: 172.20.0.1:57790 - "POST /node/69afc649275c8d3b7939b706 HTTP/1.0" 202 Accepted 2026-03-10 07:50:03,108 - lava_callback - INFO - Starting async job submit for node 69afc649275c8d3b7939b706 2026-03-10 07:50:03,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:50:03,187 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afc649275c8d3b7939b706 HTTP/1.1" 200 None 2026-03-10 07:50:03,187 - lava_callback - DEBUG - Processing node 69afc649275c8d3b7939b706: baseline-arm64-broonie 2026-03-10 07:50:03,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 07:50:03,723 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 07:50:03,724 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69afc649275c8d3b7939b706/log.txt.gz 2026-03-10 07:50:03,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 07:50:04,188 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 07:50:04,190 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69afc649275c8d3b7939b706/lava-logs.yaml 2026-03-10 07:50:04,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 07:50:04,366 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 07:50:04,368 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69afc649275c8d3b7939b706/lava_callback.json.gz DEBUG: Setup failed for 69afc649275c8d3b7939b706. Transitting job node result: incomplete -> fail 2026-03-10 07:50:04,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:50:04,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afc649275c8d3b7939b706 HTTP/1.1" 200 None 2026-03-10 07:50:04,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:50:04,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aeba41feb96b28aa34a78b HTTP/1.1" 200 None 2026-03-10 07:50:04,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:50:04,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afc649275c8d3b7939b706 HTTP/1.1" 200 None 2026-03-10 07:50:04,586 - lava_callback - INFO - Completed processing callback for node 69afc649275c8d3b7939b706 INFO: 172.20.0.1:57802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44986 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45000 - "GET / HTTP/1.0" 200 OK 2026-03-10 07:50:25,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:50:25,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:52138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38472 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42024 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42208 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45130 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44114 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36708 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46840 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37510 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37586 - "GET / HTTP/1.0" 200 OK 2026-03-10 07:59:03,218 - lava_callback - INFO - Received callback for node 69afc7d0275c8d3b7939b737 INFO: 172.20.0.1:58104 - "POST /node/69afc7d0275c8d3b7939b737 HTTP/1.0" 202 Accepted 2026-03-10 07:59:03,224 - lava_callback - INFO - Starting async job submit for node 69afc7d0275c8d3b7939b737 2026-03-10 07:59:03,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:59:03,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afc7d0275c8d3b7939b737 HTTP/1.1" 200 None 2026-03-10 07:59:03,274 - lava_callback - DEBUG - Processing node 69afc7d0275c8d3b7939b737: baseline-arm64-broonie 2026-03-10 07:59:03,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 07:59:03,761 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 07:59:03,763 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69afc7d0275c8d3b7939b737/log.txt.gz 2026-03-10 07:59:03,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 07:59:04,251 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 07:59:04,253 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69afc7d0275c8d3b7939b737/lava-logs.yaml 2026-03-10 07:59:04,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 07:59:04,435 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 07:59:04,437 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69afc7d0275c8d3b7939b737/lava_callback.json.gz DEBUG: Setup failed for 69afc7d0275c8d3b7939b737. Transitting job node result: incomplete -> fail 2026-03-10 07:59:04,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:59:04,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afc7d0275c8d3b7939b737 HTTP/1.1" 200 None 2026-03-10 07:59:04,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:59:04,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-10 07:59:04,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:59:04,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afc7d0275c8d3b7939b737 HTTP/1.1" 200 None 2026-03-10 07:59:04,639 - lava_callback - INFO - Completed processing callback for node 69afc7d0275c8d3b7939b737 INFO: 172.20.0.1:58116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54060 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54064 - "GET / HTTP/1.0" 200 OK 2026-03-10 07:59:25,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 07:59:25,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:34116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59612 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35338 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35900 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44728 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56068 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45996 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60938 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58852 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36892 - "GET / HTTP/1.0" 200 OK 2026-03-10 08:08:03,604 - lava_callback - INFO - Received callback for node 69afc81c275c8d3b7939b742 INFO: 172.20.0.1:57656 - "POST /node/69afc81c275c8d3b7939b742 HTTP/1.0" 202 Accepted 2026-03-10 08:08:03,610 - lava_callback - INFO - Starting async job submit for node 69afc81c275c8d3b7939b742 2026-03-10 08:08:03,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 08:08:03,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afc81c275c8d3b7939b742 HTTP/1.1" 200 None 2026-03-10 08:08:03,661 - lava_callback - DEBUG - Processing node 69afc81c275c8d3b7939b742: baseline-arm64-broonie 2026-03-10 08:08:04,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 08:08:04,151 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 08:08:04,152 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69afc81c275c8d3b7939b742/log.txt.gz 2026-03-10 08:08:04,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 08:08:04,677 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 08:08:04,680 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69afc81c275c8d3b7939b742/lava-logs.yaml 2026-03-10 08:08:04,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-10 08:08:04,838 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-10 08:08:04,840 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69afc81c275c8d3b7939b742/lava_callback.json.gz DEBUG: Setup failed for 69afc81c275c8d3b7939b742. Transitting job node result: incomplete -> fail 2026-03-10 08:08:04,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 08:08:04,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afc81c275c8d3b7939b742 HTTP/1.1" 200 None 2026-03-10 08:08:04,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 08:08:04,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aeba41feb96b28aa34a78b HTTP/1.1" 200 None 2026-03-10 08:08:04,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 08:08:04,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afc81c275c8d3b7939b742 HTTP/1.1" 200 None 2026-03-10 08:08:05,046 - lava_callback - INFO - Completed processing callback for node 69afc81c275c8d3b7939b742 INFO: 172.20.0.1:57658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55032 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55044 - "GET / HTTP/1.0" 200 OK 2026-03-10 08:08:25,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-10 08:08:25,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:32980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45072 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56406 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49776 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35472 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35496 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.20.0.1:34600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34604 - "GET /.well-known/security.txt HTTP/1.0" 404 Not Found INFO: 172.20.0.1:43026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55534 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55548 - "GET / HTTP/1.0" 200 OK INFO: Shutting down INFO: Waiting for application shutdown. INFO: Application shutdown complete. INFO: Finished server process [1]