INFO: Started server process [1] INFO: Waiting for application startup. INFO: Application startup complete. INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit) INFO: 172.20.0.1:48882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56884 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37640 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:12:59,116 - lava_callback - INFO - Received callback for node 69c3288218a3717aa65d019a 2026-03-27 00:12:59,122 - lava_callback - INFO - Starting async job submit for node 69c3288218a3717aa65d019a INFO: 172.20.0.1:37656 - "POST /node/69c3288218a3717aa65d019a HTTP/1.0" 202 Accepted 2026-03-27 00:12:59,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:12:59,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3288218a3717aa65d019a HTTP/1.1" 200 None 2026-03-27 00:12:59,144 - lava_callback - DEBUG - Processing node 69c3288218a3717aa65d019a: baseline-arm64-clabbe 2026-03-27 00:12:59,145 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3288218a3717aa65d019a 2026-03-27 00:12:59,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:12:59,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3288218a3717aa65d019a HTTP/1.1" 200 None 2026-03-27 00:12:59,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:12:59,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-27 00:12:59,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:12:59,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3288218a3717aa65d019a HTTP/1.1" 200 None 2026-03-27 00:12:59,222 - lava_callback - ERROR - Error processing callback for node 69c3288218a3717aa65d019a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56400 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42184 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48528 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47736 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36656 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39248 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35082 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59300 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53236 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56586 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34246 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41456 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55392 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41796 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59108 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59284 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34592 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51388 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47602 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49634 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:32:11,890 - lava_callback - INFO - Received callback for node 69c5cff42cb62322ad965e04 INFO: 172.20.0.1:32854 - "POST /node/69c5cff42cb62322ad965e04 HTTP/1.0" 202 Accepted 2026-03-27 00:32:11,901 - lava_callback - INFO - Starting async job submit for node 69c5cff42cb62322ad965e04 2026-03-27 00:32:11,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:32:11,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cff42cb62322ad965e04 HTTP/1.1" 200 None 2026-03-27 00:32:11,944 - lava_callback - DEBUG - Processing node 69c5cff42cb62322ad965e04: baseline-x86 2026-03-27 00:32:12,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:32:12,988 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:32:12,991 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c5cff42cb62322ad965e04/log.txt.gz 2026-03-27 00:32:13,140 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:32866 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:32:13,714 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:32:13,717 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c5cff42cb62322ad965e04/lava-logs.yaml 2026-03-27 00:32:13,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:32:14,230 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:32:14,232 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c5cff42cb62322ad965e04/lava_callback.json.gz 2026-03-27 00:32:14,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:32:14,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cff42cb62322ad965e04 HTTP/1.1" 200 None 2026-03-27 00:32:14,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:32:14,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc5a2cb62322ad965dce HTTP/1.1" 200 None 2026-03-27 00:32:14,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:32:14,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5cff42cb62322ad965e04 HTTP/1.1" 200 None 2026-03-27 00:32:14,369 - lava_callback - INFO - Completed processing callback for node 69c5cff42cb62322ad965e04 INFO: 172.20.0.1:32868 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32878 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:32:25,210 - lava_callback - INFO - Received callback for node 69c5cff62cb62322ad965e0d INFO: 172.20.0.1:51812 - "POST /node/69c5cff62cb62322ad965e0d HTTP/1.0" 202 Accepted 2026-03-27 00:32:25,219 - lava_callback - INFO - Starting async job submit for node 69c5cff62cb62322ad965e0d 2026-03-27 00:32:25,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:32:25,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cff62cb62322ad965e0d HTTP/1.1" 200 None 2026-03-27 00:32:25,260 - lava_callback - DEBUG - Processing node 69c5cff62cb62322ad965e0d: baseline-x86 2026-03-27 00:32:25,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:32:25,983 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:32:25,985 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c5cff62cb62322ad965e0d/log.txt.gz 2026-03-27 00:32:26,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:51818 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:32:26,660 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:32:26,663 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c5cff62cb62322ad965e0d/lava-logs.yaml 2026-03-27 00:32:26,669 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:32:27,114 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:32:27,116 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c5cff62cb62322ad965e0d/lava_callback.json.gz 2026-03-27 00:32:27,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:32:27,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cff62cb62322ad965e0d HTTP/1.1" 200 None 2026-03-27 00:32:27,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:32:27,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc5a2cb62322ad965dce HTTP/1.1" 200 None 2026-03-27 00:32:27,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:32:27,231 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5cff62cb62322ad965e0d HTTP/1.1" 200 None 2026-03-27 00:32:27,233 - lava_callback - INFO - Completed processing callback for node 69c5cff62cb62322ad965e0d 2026-03-27 00:32:29,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:32:29,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:57622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58666 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45142 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:33:26,992 - lava_callback - INFO - Received callback for node 69c32d4c18a3717aa65d0229 INFO: 172.20.0.1:45144 - "POST /node/69c32d4c18a3717aa65d0229 HTTP/1.0" 202 Accepted 2026-03-27 00:33:27,001 - lava_callback - INFO - Starting async job submit for node 69c32d4c18a3717aa65d0229 2026-03-27 00:33:27,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:33:27,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d4c18a3717aa65d0229 HTTP/1.1" 200 None 2026-03-27 00:33:27,020 - lava_callback - DEBUG - Processing node 69c32d4c18a3717aa65d0229: baseline-arm-baylibre 2026-03-27 00:33:27,020 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c32d4c18a3717aa65d0229 2026-03-27 00:33:27,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:33:27,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d4c18a3717aa65d0229 HTTP/1.1" 200 None 2026-03-27 00:33:27,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:33:27,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-27 00:33:27,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:33:27,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c32d4c18a3717aa65d0229 HTTP/1.1" 200 None 2026-03-27 00:33:27,089 - lava_callback - ERROR - Error processing callback for node 69c32d4c18a3717aa65d0229: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-27 00:33:27,654 - lava_callback - INFO - Received callback for node 69c32d4e18a3717aa65d022c INFO: 172.20.0.1:45148 - "POST /node/69c32d4e18a3717aa65d022c HTTP/1.0" 202 Accepted 2026-03-27 00:33:27,660 - lava_callback - INFO - Starting async job submit for node 69c32d4e18a3717aa65d022c 2026-03-27 00:33:27,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:33:27,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d4e18a3717aa65d022c HTTP/1.1" 200 None 2026-03-27 00:33:27,676 - lava_callback - DEBUG - Processing node 69c32d4e18a3717aa65d022c: baseline-arm-baylibre 2026-03-27 00:33:27,676 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c32d4e18a3717aa65d022c 2026-03-27 00:33:27,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:33:27,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d4e18a3717aa65d022c HTTP/1.1" 200 None 2026-03-27 00:33:27,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:33:27,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-27 00:33:27,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:33:27,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c32d4e18a3717aa65d022c HTTP/1.1" 200 None 2026-03-27 00:33:27,795 - lava_callback - ERROR - Error processing callback for node 69c32d4e18a3717aa65d022c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-27 00:33:28,333 - lava_callback - INFO - Received callback for node 69c32d4f18a3717aa65d022f INFO: 172.20.0.1:45154 - "POST /node/69c32d4f18a3717aa65d022f HTTP/1.0" 202 Accepted 2026-03-27 00:33:28,343 - lava_callback - INFO - Starting async job submit for node 69c32d4f18a3717aa65d022f 2026-03-27 00:33:28,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:33:28,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d4f18a3717aa65d022f HTTP/1.1" 200 None 2026-03-27 00:33:28,395 - lava_callback - DEBUG - Processing node 69c32d4f18a3717aa65d022f: baseline-arm-baylibre 2026-03-27 00:33:28,396 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c32d4f18a3717aa65d022f 2026-03-27 00:33:28,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:33:28,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d4f18a3717aa65d022f HTTP/1.1" 200 None 2026-03-27 00:33:28,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:33:28,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-27 00:33:28,448 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:33:28,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c32d4f18a3717aa65d022f HTTP/1.1" 200 None 2026-03-27 00:33:28,478 - lava_callback - ERROR - Error processing callback for node 69c32d4f18a3717aa65d022f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-27 00:33:29,020 - lava_callback - INFO - Received callback for node 69c32d5018a3717aa65d0233 INFO: 172.20.0.1:45158 - "POST /node/69c32d5018a3717aa65d0233 HTTP/1.0" 202 Accepted 2026-03-27 00:33:29,025 - lava_callback - INFO - Starting async job submit for node 69c32d5018a3717aa65d0233 2026-03-27 00:33:29,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:33:29,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d5018a3717aa65d0233 HTTP/1.1" 200 None 2026-03-27 00:33:29,050 - lava_callback - DEBUG - Processing node 69c32d5018a3717aa65d0233: baseline-arm-baylibre 2026-03-27 00:33:29,050 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c32d5018a3717aa65d0233 2026-03-27 00:33:29,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:33:29,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d5018a3717aa65d0233 HTTP/1.1" 200 None 2026-03-27 00:33:29,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:33:29,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-27 00:33:29,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:33:29,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c32d5018a3717aa65d0233 HTTP/1.1" 200 None 2026-03-27 00:33:29,116 - lava_callback - ERROR - Error processing callback for node 69c32d5018a3717aa65d0233: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-27 00:33:29,676 - lava_callback - INFO - Received callback for node 69c32d5318a3717aa65d023e INFO: 172.20.0.1:45168 - "POST /node/69c32d5318a3717aa65d023e HTTP/1.0" 202 Accepted 2026-03-27 00:33:29,683 - lava_callback - INFO - Starting async job submit for node 69c32d5318a3717aa65d023e 2026-03-27 00:33:29,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:33:29,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d5318a3717aa65d023e HTTP/1.1" 200 None 2026-03-27 00:33:29,700 - lava_callback - DEBUG - Processing node 69c32d5318a3717aa65d023e: baseline-arm-baylibre 2026-03-27 00:33:29,700 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c32d5318a3717aa65d023e 2026-03-27 00:33:29,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:33:29,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d5318a3717aa65d023e HTTP/1.1" 200 None 2026-03-27 00:33:29,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:33:29,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-27 00:33:29,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:33:29,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c32d5318a3717aa65d023e HTTP/1.1" 200 None 2026-03-27 00:33:29,763 - lava_callback - ERROR - Error processing callback for node 69c32d5318a3717aa65d023e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-27 00:33:30,356 - lava_callback - INFO - Received callback for node 69c32d5418a3717aa65d0243 INFO: 172.20.0.1:52034 - "POST /node/69c32d5418a3717aa65d0243 HTTP/1.0" 202 Accepted 2026-03-27 00:33:30,366 - lava_callback - INFO - Starting async job submit for node 69c32d5418a3717aa65d0243 2026-03-27 00:33:30,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:33:30,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d5418a3717aa65d0243 HTTP/1.1" 200 None 2026-03-27 00:33:30,392 - lava_callback - DEBUG - Processing node 69c32d5418a3717aa65d0243: baseline-arm-baylibre 2026-03-27 00:33:30,392 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c32d5418a3717aa65d0243 2026-03-27 00:33:30,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:33:30,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d5418a3717aa65d0243 HTTP/1.1" 200 None 2026-03-27 00:33:30,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:33:30,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-27 00:33:30,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:33:30,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c32d5418a3717aa65d0243 HTTP/1.1" 200 None 2026-03-27 00:33:30,476 - lava_callback - ERROR - Error processing callback for node 69c32d5418a3717aa65d0243: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-27 00:33:32,968 - lava_callback - INFO - Received callback for node 69c5cff52cb62322ad965e0a INFO: 172.20.0.1:52038 - "POST /node/69c5cff52cb62322ad965e0a HTTP/1.0" 202 Accepted 2026-03-27 00:33:32,978 - lava_callback - INFO - Starting async job submit for node 69c5cff52cb62322ad965e0a 2026-03-27 00:33:33,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:33:33,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cff52cb62322ad965e0a HTTP/1.1" 200 None 2026-03-27 00:33:33,040 - lava_callback - DEBUG - Processing node 69c5cff52cb62322ad965e0a: baseline-x86 2026-03-27 00:33:33,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:33:34,007 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:33:34,009 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c5cff52cb62322ad965e0a/log.txt.gz 2026-03-27 00:33:34,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:33:34,990 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:33:34,993 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c5cff52cb62322ad965e0a/lava-logs.yaml 2026-03-27 00:33:35,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:33:35,392 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:33:35,394 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c5cff52cb62322ad965e0a/lava_callback.json.gz 2026-03-27 00:33:35,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:33:35,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cff52cb62322ad965e0a HTTP/1.1" 200 None 2026-03-27 00:33:35,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:33:35,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc5a2cb62322ad965dce HTTP/1.1" 200 None 2026-03-27 00:33:35,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:33:35,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5cff52cb62322ad965e0a HTTP/1.1" 200 None 2026-03-27 00:33:35,523 - lava_callback - INFO - Completed processing callback for node 69c5cff52cb62322ad965e0a INFO: 172.20.0.1:52052 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:33:40,135 - lava_callback - INFO - Received callback for node 69c32d5c18a3717aa65d025c INFO: 172.20.0.1:57790 - "POST /node/69c32d5c18a3717aa65d025c HTTP/1.0" 202 Accepted 2026-03-27 00:33:40,144 - lava_callback - INFO - Starting async job submit for node 69c32d5c18a3717aa65d025c 2026-03-27 00:33:40,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:33:40,170 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d5c18a3717aa65d025c HTTP/1.1" 200 None 2026-03-27 00:33:40,171 - lava_callback - DEBUG - Processing node 69c32d5c18a3717aa65d025c: baseline-arm-clabbe 2026-03-27 00:33:40,171 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c32d5c18a3717aa65d025c 2026-03-27 00:33:40,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:33:40,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d5c18a3717aa65d025c HTTP/1.1" 200 None 2026-03-27 00:33:40,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:33:40,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-27 00:33:40,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:33:40,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c32d5c18a3717aa65d025c HTTP/1.1" 200 None 2026-03-27 00:33:40,304 - lava_callback - ERROR - Error processing callback for node 69c32d5c18a3717aa65d025c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-27 00:33:40,833 - lava_callback - INFO - Received callback for node 69c32d6018a3717aa65d0269 INFO: 172.20.0.1:57802 - "POST /node/69c32d6018a3717aa65d0269 HTTP/1.0" 202 Accepted 2026-03-27 00:33:40,845 - lava_callback - INFO - Starting async job submit for node 69c32d6018a3717aa65d0269 2026-03-27 00:33:40,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:33:40,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d6018a3717aa65d0269 HTTP/1.1" 200 None 2026-03-27 00:33:40,882 - lava_callback - DEBUG - Processing node 69c32d6018a3717aa65d0269: baseline-arm-clabbe 2026-03-27 00:33:40,882 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c32d6018a3717aa65d0269 2026-03-27 00:33:40,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:33:40,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d6018a3717aa65d0269 HTTP/1.1" 200 None 2026-03-27 00:33:40,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:33:40,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-27 00:33:40,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:33:40,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c32d6018a3717aa65d0269 HTTP/1.1" 200 None 2026-03-27 00:33:40,992 - lava_callback - ERROR - Error processing callback for node 69c32d6018a3717aa65d0269: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-27 00:33:41,523 - lava_callback - INFO - Received callback for node 69c32d6118a3717aa65d026c INFO: 172.20.0.1:57814 - "POST /node/69c32d6118a3717aa65d026c HTTP/1.0" 202 Accepted 2026-03-27 00:33:41,533 - lava_callback - INFO - Starting async job submit for node 69c32d6118a3717aa65d026c 2026-03-27 00:33:41,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:33:41,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d6118a3717aa65d026c HTTP/1.1" 200 None 2026-03-27 00:33:41,604 - lava_callback - DEBUG - Processing node 69c32d6118a3717aa65d026c: baseline-arm-clabbe 2026-03-27 00:33:41,604 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c32d6118a3717aa65d026c 2026-03-27 00:33:41,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:33:41,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d6118a3717aa65d026c HTTP/1.1" 200 None 2026-03-27 00:33:41,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:33:41,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-27 00:33:41,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:33:41,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c32d6118a3717aa65d026c HTTP/1.1" 200 None 2026-03-27 00:33:41,723 - lava_callback - ERROR - Error processing callback for node 69c32d6118a3717aa65d026c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-27 00:33:42,200 - lava_callback - INFO - Received callback for node 69c32d6218a3717aa65d026f INFO: 172.20.0.1:57816 - "POST /node/69c32d6218a3717aa65d026f HTTP/1.0" 202 Accepted 2026-03-27 00:33:42,209 - lava_callback - INFO - Starting async job submit for node 69c32d6218a3717aa65d026f 2026-03-27 00:33:42,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:33:42,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d6218a3717aa65d026f HTTP/1.1" 200 None 2026-03-27 00:33:42,242 - lava_callback - DEBUG - Processing node 69c32d6218a3717aa65d026f: baseline-arm-clabbe 2026-03-27 00:33:42,243 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c32d6218a3717aa65d026f 2026-03-27 00:33:42,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:33:42,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d6218a3717aa65d026f HTTP/1.1" 200 None 2026-03-27 00:33:42,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:33:42,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-27 00:33:42,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:33:42,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c32d6218a3717aa65d026f HTTP/1.1" 200 None 2026-03-27 00:33:42,393 - lava_callback - ERROR - Error processing callback for node 69c32d6218a3717aa65d026f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57818 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:33:46,822 - lava_callback - INFO - Received callback for node 69c5d0572cb62322ad965e58 INFO: 172.20.0.1:57830 - "POST /node/69c5d0572cb62322ad965e58 HTTP/1.0" 202 Accepted 2026-03-27 00:33:46,833 - lava_callback - INFO - Starting async job submit for node 69c5d0572cb62322ad965e58 2026-03-27 00:33:46,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:33:46,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d0572cb62322ad965e58 HTTP/1.1" 200 None 2026-03-27 00:33:46,865 - lava_callback - DEBUG - Processing node 69c5d0572cb62322ad965e58: baseline-arm 2026-03-27 00:33:46,874 - lava_callback - WARNING - Failed to upload log 2026-03-27 00:33:46,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:33:47,449 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:33:47,452 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c5d0572cb62322ad965e58/lava-logs.yaml 2026-03-27 00:33:47,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:33:47,923 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:33:47,925 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c5d0572cb62322ad965e58/lava_callback.json.gz 2026-03-27 00:33:47,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:33:47,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d0572cb62322ad965e58 HTTP/1.1" 200 None 2026-03-27 00:33:47,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:33:48,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-27 00:33:48,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:33:48,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d0572cb62322ad965e58 HTTP/1.1" 200 None 2026-03-27 00:33:48,061 - lava_callback - INFO - Completed processing callback for node 69c5d0572cb62322ad965e58 2026-03-27 00:33:50,449 - lava_callback - INFO - Received callback for node 69c5d0562cb62322ad965e4a INFO: 172.20.0.1:43354 - "POST /node/69c5d0562cb62322ad965e4a HTTP/1.0" 202 Accepted 2026-03-27 00:33:50,467 - lava_callback - INFO - Starting async job submit for node 69c5d0562cb62322ad965e4a 2026-03-27 00:33:50,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:33:50,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d0562cb62322ad965e4a HTTP/1.1" 200 None 2026-03-27 00:33:50,527 - lava_callback - DEBUG - Processing node 69c5d0562cb62322ad965e4a: baseline-arm 2026-03-27 00:33:50,550 - lava_callback - WARNING - Failed to upload log 2026-03-27 00:33:50,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:33:50,922 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:33:50,925 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c5d0562cb62322ad965e4a/lava-logs.yaml 2026-03-27 00:33:50,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:33:51,338 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:33:51,341 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c5d0562cb62322ad965e4a/lava_callback.json.gz 2026-03-27 00:33:51,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:33:51,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d0562cb62322ad965e4a HTTP/1.1" 200 None 2026-03-27 00:33:51,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:33:51,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-27 00:33:51,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:33:51,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d0562cb62322ad965e4a HTTP/1.1" 200 None 2026-03-27 00:33:51,465 - lava_callback - INFO - Completed processing callback for node 69c5d0562cb62322ad965e4a INFO: 172.20.0.1:43368 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:33:59,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:33:59,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-27 00:34:00,113 - lava_callback - INFO - Received callback for node 69c32d6518a3717aa65d0278 INFO: 172.20.0.1:45658 - "POST /node/69c32d6518a3717aa65d0278 HTTP/1.0" 202 Accepted 2026-03-27 00:34:00,128 - lava_callback - INFO - Starting async job submit for node 69c32d6518a3717aa65d0278 2026-03-27 00:34:00,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:34:00,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d6518a3717aa65d0278 HTTP/1.1" 200 None 2026-03-27 00:34:00,221 - lava_callback - DEBUG - Processing node 69c32d6518a3717aa65d0278: baseline-arm-clabbe 2026-03-27 00:34:00,221 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c32d6518a3717aa65d0278 2026-03-27 00:34:00,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:34:00,292 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d6518a3717aa65d0278 HTTP/1.1" 200 None 2026-03-27 00:34:00,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:34:00,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-27 00:34:00,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:34:00,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c32d6518a3717aa65d0278 HTTP/1.1" 200 None 2026-03-27 00:34:00,445 - lava_callback - ERROR - Error processing callback for node 69c32d6518a3717aa65d0278: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39816 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:34:13,561 - lava_callback - INFO - Received callback for node 69c5d05a2cb62322ad965e6b INFO: 172.20.0.1:39822 - "POST /node/69c5d05a2cb62322ad965e6b HTTP/1.0" 202 Accepted 2026-03-27 00:34:13,618 - lava_callback - INFO - Starting async job submit for node 69c5d05a2cb62322ad965e6b 2026-03-27 00:34:13,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:34:13,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d05a2cb62322ad965e6b HTTP/1.1" 200 None 2026-03-27 00:34:13,681 - lava_callback - DEBUG - Processing node 69c5d05a2cb62322ad965e6b: baseline-arm 2026-03-27 00:34:13,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:34:14,315 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:34:14,317 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c5d05a2cb62322ad965e6b/log.txt.gz 2026-03-27 00:34:14,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:34:15,142 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:34:15,144 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c5d05a2cb62322ad965e6b/lava-logs.yaml 2026-03-27 00:34:15,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:39826 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-27 00:34:15,468 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:34:15,470 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c5d05a2cb62322ad965e6b/lava_callback.json.gz 2026-03-27 00:34:15,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:34:15,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d05a2cb62322ad965e6b HTTP/1.1" 200 None 2026-03-27 00:34:15,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:34:15,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-27 00:34:15,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:34:15,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d05a2cb62322ad965e6b HTTP/1.1" 200 None 2026-03-27 00:34:15,616 - lava_callback - INFO - Completed processing callback for node 69c5d05a2cb62322ad965e6b INFO: 172.20.0.1:39828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53870 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:34:29,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:34:29,327 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:36674 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:34:45,462 - lava_callback - INFO - Received callback for node 69c5d0592cb62322ad965e64 INFO: 172.20.0.1:56556 - "POST /node/69c5d0592cb62322ad965e64 HTTP/1.0" 202 Accepted 2026-03-27 00:34:45,468 - lava_callback - INFO - Starting async job submit for node 69c5d0592cb62322ad965e64 2026-03-27 00:34:45,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:34:45,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d0592cb62322ad965e64 HTTP/1.1" 200 None 2026-03-27 00:34:45,505 - lava_callback - DEBUG - Processing node 69c5d0592cb62322ad965e64: baseline-arm 2026-03-27 00:34:45,742 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:34:45,912 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:34:45,914 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c5d0592cb62322ad965e64/log.txt.gz 2026-03-27 00:34:46,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:34:46,275 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:34:46,278 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c5d0592cb62322ad965e64/lava-logs.yaml 2026-03-27 00:34:46,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:34:46,430 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:34:46,432 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c5d0592cb62322ad965e64/lava_callback.json.gz 2026-03-27 00:34:46,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:34:46,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d0592cb62322ad965e64 HTTP/1.1" 200 None 2026-03-27 00:34:46,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:34:46,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-27 00:34:46,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:56558 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:34:46,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d0592cb62322ad965e64 HTTP/1.1" 200 None 2026-03-27 00:34:46,520 - lava_callback - INFO - Completed processing callback for node 69c5d0592cb62322ad965e64 2026-03-27 00:34:52,936 - lava_callback - INFO - Received callback for node 69c5d0562cb62322ad965e47 INFO: 172.20.0.1:41410 - "POST /node/69c5d0562cb62322ad965e47 HTTP/1.0" 202 Accepted 2026-03-27 00:34:52,947 - lava_callback - INFO - Starting async job submit for node 69c5d0562cb62322ad965e47 2026-03-27 00:34:52,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:34:52,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d0562cb62322ad965e47 HTTP/1.1" 200 None 2026-03-27 00:34:52,990 - lava_callback - DEBUG - Processing node 69c5d0562cb62322ad965e47: baseline-arm 2026-03-27 00:34:53,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:34:53,294 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:34:53,295 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c5d0562cb62322ad965e47/log.txt.gz 2026-03-27 00:34:53,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:34:53,577 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:34:53,578 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c5d0562cb62322ad965e47/lava-logs.yaml 2026-03-27 00:34:53,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:34:53,745 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:34:53,747 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c5d0562cb62322ad965e47/lava_callback.json.gz 2026-03-27 00:34:53,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:34:53,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d0562cb62322ad965e47 HTTP/1.1" 200 None 2026-03-27 00:34:53,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:34:53,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-27 00:34:53,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:34:53,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d0562cb62322ad965e47 HTTP/1.1" 200 None 2026-03-27 00:34:53,853 - lava_callback - INFO - Completed processing callback for node 69c5d0562cb62322ad965e47 INFO: 172.20.0.1:41412 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:34:59,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:34:59,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:39244 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:35:06,907 - lava_callback - INFO - Received callback for node 69c5d0832cb62322ad965ef9 INFO: 172.20.0.1:39254 - "POST /node/69c5d0832cb62322ad965ef9 HTTP/1.0" 202 Accepted 2026-03-27 00:35:06,913 - lava_callback - INFO - Starting async job submit for node 69c5d0832cb62322ad965ef9 2026-03-27 00:35:06,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:35:06,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d0832cb62322ad965ef9 HTTP/1.1" 200 None 2026-03-27 00:35:06,966 - lava_callback - DEBUG - Processing node 69c5d0832cb62322ad965ef9: baseline-arm-pengutronix 2026-03-27 00:35:07,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:35:07,252 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:35:07,254 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c5d0832cb62322ad965ef9/log.txt.gz 2026-03-27 00:35:07,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:35:07,539 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:35:07,541 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c5d0832cb62322ad965ef9/lava-logs.yaml 2026-03-27 00:35:07,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:35:07,679 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:35:07,681 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c5d0832cb62322ad965ef9/lava_callback.json.gz 2026-03-27 00:35:07,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:35:07,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d0832cb62322ad965ef9 HTTP/1.1" 200 None 2026-03-27 00:35:07,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:35:07,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-27 00:35:07,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:35:07,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d0832cb62322ad965ef9 HTTP/1.1" 200 None 2026-03-27 00:35:07,771 - lava_callback - INFO - Completed processing callback for node 69c5d0832cb62322ad965ef9 2026-03-27 00:35:07,906 - lava_callback - INFO - Received callback for node 69c5d0822cb62322ad965ef6 INFO: 172.20.0.1:39260 - "POST /node/69c5d0822cb62322ad965ef6 HTTP/1.0" 202 Accepted 2026-03-27 00:35:07,917 - lava_callback - INFO - Starting async job submit for node 69c5d0822cb62322ad965ef6 2026-03-27 00:35:07,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:35:07,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d0822cb62322ad965ef6 HTTP/1.1" 200 None 2026-03-27 00:35:07,966 - lava_callback - DEBUG - Processing node 69c5d0822cb62322ad965ef6: baseline-arm-pengutronix 2026-03-27 00:35:08,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:35:08,285 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:35:08,287 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c5d0822cb62322ad965ef6/log.txt.gz 2026-03-27 00:35:08,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:35:08,580 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:35:08,582 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c5d0822cb62322ad965ef6/lava-logs.yaml 2026-03-27 00:35:08,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:35:08,739 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:35:08,741 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c5d0822cb62322ad965ef6/lava_callback.json.gz 2026-03-27 00:35:08,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:35:08,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d0822cb62322ad965ef6 HTTP/1.1" 200 None 2026-03-27 00:35:08,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:35:08,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-27 00:35:08,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:35:08,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d0822cb62322ad965ef6 HTTP/1.1" 200 None 2026-03-27 00:35:08,818 - lava_callback - INFO - Completed processing callback for node 69c5d0822cb62322ad965ef6 INFO: 172.20.0.1:48868 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:35:13,371 - lava_callback - INFO - Received callback for node 69c5d0442cb62322ad965e39 INFO: 172.20.0.1:48870 - "POST /node/69c5d0442cb62322ad965e39 HTTP/1.0" 202 Accepted 2026-03-27 00:35:13,380 - lava_callback - INFO - Starting async job submit for node 69c5d0442cb62322ad965e39 2026-03-27 00:35:13,405 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:35:13,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d0442cb62322ad965e39 HTTP/1.1" 200 None 2026-03-27 00:35:13,420 - lava_callback - DEBUG - Processing node 69c5d0442cb62322ad965e39: baseline-arm 2026-03-27 00:35:13,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:35:13,733 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:35:13,735 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c5d0442cb62322ad965e39/log.txt.gz 2026-03-27 00:35:13,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:35:14,030 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:35:14,032 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c5d0442cb62322ad965e39/lava-logs.yaml 2026-03-27 00:35:14,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:35:14,167 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:35:14,169 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c5d0442cb62322ad965e39/lava_callback.json.gz 2026-03-27 00:35:14,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:35:14,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d0442cb62322ad965e39 HTTP/1.1" 200 None 2026-03-27 00:35:14,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:35:14,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc312cb62322ad965dad HTTP/1.1" 200 None 2026-03-27 00:35:14,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:35:14,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d0442cb62322ad965e39 HTTP/1.1" 200 None 2026-03-27 00:35:14,307 - lava_callback - INFO - Completed processing callback for node 69c5d0442cb62322ad965e39 2026-03-27 00:35:15,023 - lava_callback - INFO - Received callback for node 69c5d06b2cb62322ad965e97 INFO: 172.20.0.1:48880 - "POST /node/69c5d06b2cb62322ad965e97 HTTP/1.0" 202 Accepted 2026-03-27 00:35:15,029 - lava_callback - INFO - Starting async job submit for node 69c5d06b2cb62322ad965e97 2026-03-27 00:35:15,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:35:15,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d06b2cb62322ad965e97 HTTP/1.1" 200 None 2026-03-27 00:35:15,066 - lava_callback - DEBUG - Processing node 69c5d06b2cb62322ad965e97: baseline-arm-broonie INFO: 172.20.0.1:48886 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-27 00:35:15,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:35:15,432 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:35:15,433 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c5d06b2cb62322ad965e97/log.txt.gz 2026-03-27 00:35:15,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:35:15,760 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:35:15,762 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c5d06b2cb62322ad965e97/lava-logs.yaml 2026-03-27 00:35:15,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:35:15,913 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:35:15,915 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c5d06b2cb62322ad965e97/lava_callback.json.gz 2026-03-27 00:35:15,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:35:15,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d06b2cb62322ad965e97 HTTP/1.1" 200 None 2026-03-27 00:35:15,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:35:15,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-27 00:35:15,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:35:15,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d06b2cb62322ad965e97 HTTP/1.1" 200 None 2026-03-27 00:35:15,994 - lava_callback - INFO - Completed processing callback for node 69c5d06b2cb62322ad965e97 2026-03-27 00:35:16,171 - lava_callback - INFO - Received callback for node 69c5d0862cb62322ad965eff INFO: 172.20.0.1:48898 - "POST /node/69c5d0862cb62322ad965eff HTTP/1.0" 202 Accepted 2026-03-27 00:35:16,218 - lava_callback - INFO - Starting async job submit for node 69c5d0862cb62322ad965eff 2026-03-27 00:35:16,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:35:16,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d0862cb62322ad965eff HTTP/1.1" 200 None 2026-03-27 00:35:16,263 - lava_callback - DEBUG - Processing node 69c5d0862cb62322ad965eff: baseline-arm-pengutronix 2026-03-27 00:35:16,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:35:16,532 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:35:16,534 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c5d0862cb62322ad965eff/log.txt.gz INFO: 172.20.0.1:48900 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:35:16,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:35:16,761 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:35:16,763 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c5d0862cb62322ad965eff/lava-logs.yaml 2026-03-27 00:35:16,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:35:16,879 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:35:16,881 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c5d0862cb62322ad965eff/lava_callback.json.gz 2026-03-27 00:35:16,902 - lava_callback - INFO - Received callback for node 69c5d06d2cb62322ad965e9c 2026-03-27 00:35:16,919 - lava_callback - INFO - Starting async job submit for node 69c5d06d2cb62322ad965e9c INFO: 172.20.0.1:48914 - "POST /node/69c5d06d2cb62322ad965e9c HTTP/1.0" 202 Accepted 2026-03-27 00:35:16,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:35:16,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:35:16,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d06d2cb62322ad965e9c HTTP/1.1" 200 None 2026-03-27 00:35:16,962 - lava_callback - DEBUG - Processing node 69c5d06d2cb62322ad965e9c: baseline-arm-broonie 2026-03-27 00:35:17,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d0862cb62322ad965eff HTTP/1.1" 200 None 2026-03-27 00:35:17,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:35:17,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-27 00:35:17,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:35:17,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:35:17,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d0862cb62322ad965eff HTTP/1.1" 200 None 2026-03-27 00:35:17,214 - lava_callback - INFO - Completed processing callback for node 69c5d0862cb62322ad965eff 2026-03-27 00:35:17,353 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:35:17,354 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c5d06d2cb62322ad965e9c/log.txt.gz 2026-03-27 00:35:17,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:35:17,670 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:35:17,672 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c5d06d2cb62322ad965e9c/lava-logs.yaml 2026-03-27 00:35:17,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:35:17,823 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:35:17,825 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c5d06d2cb62322ad965e9c/lava_callback.json.gz 2026-03-27 00:35:17,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:35:17,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d06d2cb62322ad965e9c HTTP/1.1" 200 None 2026-03-27 00:35:17,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:35:17,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-27 00:35:17,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:35:17,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d06d2cb62322ad965e9c HTTP/1.1" 200 None 2026-03-27 00:35:17,934 - lava_callback - INFO - Completed processing callback for node 69c5d06d2cb62322ad965e9c 2026-03-27 00:35:18,790 - lava_callback - INFO - Received callback for node 69c5d0742cb62322ad965eb0 INFO: 172.20.0.1:48930 - "POST /node/69c5d0742cb62322ad965eb0 HTTP/1.0" 202 Accepted 2026-03-27 00:35:18,796 - lava_callback - INFO - Starting async job submit for node 69c5d0742cb62322ad965eb0 2026-03-27 00:35:18,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:35:18,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d0742cb62322ad965eb0 HTTP/1.1" 200 None 2026-03-27 00:35:18,834 - lava_callback - DEBUG - Processing node 69c5d0742cb62322ad965eb0: baseline-arm-clabbe 2026-03-27 00:35:19,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:35:19,180 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:35:19,182 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c5d0742cb62322ad965eb0/log.txt.gz 2026-03-27 00:35:19,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:35:19,542 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:35:19,544 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c5d0742cb62322ad965eb0/lava-logs.yaml 2026-03-27 00:35:19,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:35:19,683 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:35:19,684 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c5d0742cb62322ad965eb0/lava_callback.json.gz 2026-03-27 00:35:19,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:35:19,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d0742cb62322ad965eb0 HTTP/1.1" 200 None 2026-03-27 00:35:19,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:35:19,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-27 00:35:19,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:35:19,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d0742cb62322ad965eb0 HTTP/1.1" 200 None 2026-03-27 00:35:19,769 - lava_callback - INFO - Completed processing callback for node 69c5d0742cb62322ad965eb0 INFO: 172.20.0.1:57306 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:35:29,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:35:29,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-27 00:35:33,260 - lava_callback - INFO - Received callback for node 69c5d0692cb62322ad965e94 INFO: 172.20.0.1:52182 - "POST /node/69c5d0692cb62322ad965e94 HTTP/1.0" 202 Accepted 2026-03-27 00:35:33,266 - lava_callback - INFO - Starting async job submit for node 69c5d0692cb62322ad965e94 2026-03-27 00:35:33,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:35:33,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d0692cb62322ad965e94 HTTP/1.1" 200 None 2026-03-27 00:35:33,304 - lava_callback - DEBUG - Processing node 69c5d0692cb62322ad965e94: baseline-arm-broonie 2026-03-27 00:35:33,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:35:33,644 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:35:33,646 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c5d0692cb62322ad965e94/log.txt.gz 2026-03-27 00:35:33,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:35:33,947 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:35:33,949 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c5d0692cb62322ad965e94/lava-logs.yaml 2026-03-27 00:35:33,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:35:34,078 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:35:34,079 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c5d0692cb62322ad965e94/lava_callback.json.gz 2026-03-27 00:35:34,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:35:34,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d0692cb62322ad965e94 HTTP/1.1" 200 None 2026-03-27 00:35:34,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:35:34,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-27 00:35:34,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:35:34,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d0692cb62322ad965e94 HTTP/1.1" 200 None 2026-03-27 00:35:34,171 - lava_callback - INFO - Completed processing callback for node 69c5d0692cb62322ad965e94 INFO: 172.20.0.1:52196 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:35:45,409 - lava_callback - INFO - Received callback for node 69c5d06f2cb62322ad965e9f INFO: 172.20.0.1:57774 - "POST /node/69c5d06f2cb62322ad965e9f HTTP/1.0" 202 Accepted 2026-03-27 00:35:45,415 - lava_callback - INFO - Starting async job submit for node 69c5d06f2cb62322ad965e9f 2026-03-27 00:35:45,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:35:45,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d06f2cb62322ad965e9f HTTP/1.1" 200 None 2026-03-27 00:35:45,454 - lava_callback - DEBUG - Processing node 69c5d06f2cb62322ad965e9f: baseline-arm-broonie 2026-03-27 00:35:45,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:35:45,824 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:35:45,826 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c5d06f2cb62322ad965e9f/log.txt.gz 2026-03-27 00:35:45,971 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:35:46,192 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:35:46,194 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c5d06f2cb62322ad965e9f/lava-logs.yaml 2026-03-27 00:35:46,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:35:46,342 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:35:46,343 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c5d06f2cb62322ad965e9f/lava_callback.json.gz 2026-03-27 00:35:46,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:35:46,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d06f2cb62322ad965e9f HTTP/1.1" 200 None 2026-03-27 00:35:46,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:35:46,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-27 00:35:46,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:35:46,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d06f2cb62322ad965e9f HTTP/1.1" 200 None 2026-03-27 00:35:46,424 - lava_callback - INFO - Completed processing callback for node 69c5d06f2cb62322ad965e9f INFO: 172.20.0.1:57790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49576 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:35:59,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:35:59,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:49980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36494 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60052 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:36:34,505 - lava_callback - INFO - Received callback for node 69c5d09f2cb62322ad965f39 INFO: 172.20.0.1:57778 - "POST /node/69c5d09f2cb62322ad965f39 HTTP/1.0" 202 Accepted 2026-03-27 00:36:34,512 - lava_callback - INFO - Starting async job submit for node 69c5d09f2cb62322ad965f39 2026-03-27 00:36:34,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:36:34,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d09f2cb62322ad965f39 HTTP/1.1" 200 None 2026-03-27 00:36:34,550 - lava_callback - DEBUG - Processing node 69c5d09f2cb62322ad965f39: ltp-dio 2026-03-27 00:36:34,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:36:35,116 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:36:35,118 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-dio-69c5d09f2cb62322ad965f39/log.txt.gz 2026-03-27 00:36:35,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:36:35,558 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:36:35,560 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-dio-69c5d09f2cb62322ad965f39/lava-logs.yaml 2026-03-27 00:36:35,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:36:35,916 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:36:35,917 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-dio-69c5d09f2cb62322ad965f39/lava_callback.json.gz 2026-03-27 00:36:35,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:36:35,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d09f2cb62322ad965f39 HTTP/1.1" 200 None 2026-03-27 00:36:35,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:36:35,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-27 00:36:35,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:36:35,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d09f2cb62322ad965f39 HTTP/1.1" 200 None 2026-03-27 00:36:35,998 - lava_callback - INFO - Completed processing callback for node 69c5d09f2cb62322ad965f39 INFO: 172.20.0.1:57784 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:36:45,801 - lava_callback - INFO - Received callback for node 69c5d0752cb62322ad965ecc INFO: 172.20.0.1:57718 - "POST /node/69c5d0752cb62322ad965ecc HTTP/1.0" 202 Accepted 2026-03-27 00:36:45,807 - lava_callback - INFO - Starting async job submit for node 69c5d0752cb62322ad965ecc 2026-03-27 00:36:45,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:36:45,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d0752cb62322ad965ecc HTTP/1.1" 200 None 2026-03-27 00:36:45,844 - lava_callback - DEBUG - Processing node 69c5d0752cb62322ad965ecc: baseline-arm-clabbe 2026-03-27 00:36:46,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:36:46,182 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:36:46,184 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c5d0752cb62322ad965ecc/log.txt.gz 2026-03-27 00:36:46,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:36:46,501 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:36:46,502 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c5d0752cb62322ad965ecc/lava-logs.yaml 2026-03-27 00:36:46,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:57722 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:36:46,664 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:36:46,666 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c5d0752cb62322ad965ecc/lava_callback.json.gz 2026-03-27 00:36:46,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:36:46,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d0752cb62322ad965ecc HTTP/1.1" 200 None 2026-03-27 00:36:46,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:36:46,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-27 00:36:46,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:36:46,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d0752cb62322ad965ecc HTTP/1.1" 200 None 2026-03-27 00:36:46,754 - lava_callback - INFO - Completed processing callback for node 69c5d0752cb62322ad965ecc INFO: 172.20.0.1:57736 - "GET /node/6985bc7e71989fb14149758d%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:55454 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:36:59,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:36:59,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:55062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55642 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38958 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:37:44,512 - lava_callback - INFO - Received callback for node 69c5c597abf66a8f4d9bd25e INFO: 172.20.0.1:56714 - "POST /node/69c5c597abf66a8f4d9bd25e HTTP/1.0" 202 Accepted 2026-03-27 00:37:44,521 - lava_callback - INFO - Starting async job submit for node 69c5c597abf66a8f4d9bd25e 2026-03-27 00:37:44,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:37:44,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5c597abf66a8f4d9bd25e HTTP/1.1" 200 None 2026-03-27 00:37:44,583 - lava_callback - DEBUG - Processing node 69c5c597abf66a8f4d9bd25e: baseline-nfs-arm64 2026-03-27 00:37:45,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:37:45,910 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:37:45,911 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5c597abf66a8f4d9bd25e/log.txt.gz INFO: 172.20.0.1:56730 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:37:46,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:37:46,830 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:37:46,832 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5c597abf66a8f4d9bd25e/lava-logs.yaml 2026-03-27 00:37:46,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:37:47,050 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:37:47,052 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5c597abf66a8f4d9bd25e/lava_callback.json.gz DEBUG: Setup failed for 69c5c597abf66a8f4d9bd25e. Transitting job node result: incomplete -> fail 2026-03-27 00:37:47,140 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:37:47,158 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5c597abf66a8f4d9bd25e HTTP/1.1" 200 None 2026-03-27 00:37:47,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:37:47,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-27 00:37:47,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:37:47,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5c597abf66a8f4d9bd25e HTTP/1.1" 200 None 2026-03-27 00:37:47,297 - lava_callback - INFO - Completed processing callback for node 69c5c597abf66a8f4d9bd25e INFO: 172.20.0.1:57580 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:37:59,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:37:59,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:44858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37104 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47024 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:38:28,877 - lava_callback - INFO - Received callback for node 69c5d07a2cb62322ad965ee1 INFO: 172.20.0.1:47028 - "POST /node/69c5d07a2cb62322ad965ee1 HTTP/1.0" 202 Accepted 2026-03-27 00:38:28,883 - lava_callback - INFO - Starting async job submit for node 69c5d07a2cb62322ad965ee1 2026-03-27 00:38:28,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:38:28,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d07a2cb62322ad965ee1 HTTP/1.1" 200 None 2026-03-27 00:38:28,923 - lava_callback - DEBUG - Processing node 69c5d07a2cb62322ad965ee1: baseline-arm-clabbe 2026-03-27 00:38:29,147 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:38:29,291 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:38:29,293 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c5d07a2cb62322ad965ee1/log.txt.gz 2026-03-27 00:38:29,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:38:29,639 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:38:29,640 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c5d07a2cb62322ad965ee1/lava-logs.yaml 2026-03-27 00:38:29,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:38:29,794 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:38:29,795 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c5d07a2cb62322ad965ee1/lava_callback.json.gz 2026-03-27 00:38:29,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:38:29,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d07a2cb62322ad965ee1 HTTP/1.1" 200 None 2026-03-27 00:38:29,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:38:29,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-27 00:38:29,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:38:29,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d07a2cb62322ad965ee1 HTTP/1.1" 200 None 2026-03-27 00:38:29,878 - lava_callback - INFO - Completed processing callback for node 69c5d07a2cb62322ad965ee1 INFO: 172.20.0.1:43944 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:38:43,827 - lava_callback - INFO - Received callback for node 69c5d0972cb62322ad965f2d INFO: 172.20.0.1:47752 - "POST /node/69c5d0972cb62322ad965f2d HTTP/1.0" 202 Accepted 2026-03-27 00:38:43,836 - lava_callback - INFO - Starting async job submit for node 69c5d0972cb62322ad965f2d 2026-03-27 00:38:43,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:38:43,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d0972cb62322ad965f2d HTTP/1.1" 200 None 2026-03-27 00:38:43,879 - lava_callback - DEBUG - Processing node 69c5d0972cb62322ad965f2d: ltp-cap_bounds 2026-03-27 00:38:44,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:38:44,493 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:38:44,495 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69c5d0972cb62322ad965f2d/log.txt.gz 2026-03-27 00:38:44,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:38:44,884 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:38:44,886 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69c5d0972cb62322ad965f2d/lava-logs.yaml 2026-03-27 00:38:44,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:38:45,082 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:38:45,084 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69c5d0972cb62322ad965f2d/lava_callback.json.gz 2026-03-27 00:38:45,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:38:45,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d0972cb62322ad965f2d HTTP/1.1" 200 None 2026-03-27 00:38:45,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:38:45,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-27 00:38:45,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:38:45,164 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d0972cb62322ad965f2d HTTP/1.1" 200 None 2026-03-27 00:38:45,174 - lava_callback - INFO - Completed processing callback for node 69c5d0972cb62322ad965f2d 2026-03-27 00:38:45,465 - lava_callback - INFO - Received callback for node 69c5d0992cb62322ad965f30 INFO: 172.20.0.1:47756 - "POST /node/69c5d0992cb62322ad965f30 HTTP/1.0" 202 Accepted 2026-03-27 00:38:45,475 - lava_callback - INFO - Starting async job submit for node 69c5d0992cb62322ad965f30 2026-03-27 00:38:45,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:38:45,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d0992cb62322ad965f30 HTTP/1.1" 200 None 2026-03-27 00:38:45,521 - lava_callback - DEBUG - Processing node 69c5d0992cb62322ad965f30: ltp-containers 2026-03-27 00:38:45,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:38:46,037 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:38:46,039 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-containers-69c5d0992cb62322ad965f30/log.txt.gz 2026-03-27 00:38:46,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:38:46,439 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:38:46,441 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-containers-69c5d0992cb62322ad965f30/lava-logs.yaml 2026-03-27 00:38:46,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:47772 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:38:46,590 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:38:46,592 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-containers-69c5d0992cb62322ad965f30/lava_callback.json.gz 2026-03-27 00:38:46,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:38:46,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d0992cb62322ad965f30 HTTP/1.1" 200 None 2026-03-27 00:38:46,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:38:46,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-27 00:38:46,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:38:46,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d0992cb62322ad965f30 HTTP/1.1" 200 None 2026-03-27 00:38:46,683 - lava_callback - INFO - Completed processing callback for node 69c5d0992cb62322ad965f30 INFO: 172.20.0.1:35418 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:38:59,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:38:59,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-27 00:39:04,080 - lava_callback - INFO - Received callback for node 69c5d0ab2cb62322ad965f57 INFO: 172.20.0.1:39130 - "POST /node/69c5d0ab2cb62322ad965f57 HTTP/1.0" 202 Accepted 2026-03-27 00:39:04,087 - lava_callback - INFO - Starting async job submit for node 69c5d0ab2cb62322ad965f57 2026-03-27 00:39:04,112 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:39:04,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d0ab2cb62322ad965f57 HTTP/1.1" 200 None 2026-03-27 00:39:04,134 - lava_callback - DEBUG - Processing node 69c5d0ab2cb62322ad965f57: ltp-syscalls 2026-03-27 00:39:04,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:39:05,094 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:39:05,095 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-69c5d0ab2cb62322ad965f57/log.txt.gz 2026-03-27 00:39:05,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:39:05,725 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:39:05,727 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-69c5d0ab2cb62322ad965f57/lava-logs.yaml 2026-03-27 00:39:05,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:39:05,890 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:39:05,892 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-69c5d0ab2cb62322ad965f57/lava_callback.json.gz 2026-03-27 00:39:05,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:39:05,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d0ab2cb62322ad965f57 HTTP/1.1" 200 None 2026-03-27 00:39:05,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:39:05,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-27 00:39:05,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:39:05,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d0ab2cb62322ad965f57 HTTP/1.1" 200 None 2026-03-27 00:39:05,981 - lava_callback - INFO - Completed processing callback for node 69c5d0ab2cb62322ad965f57 INFO: 172.20.0.1:39136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45574 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45576 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:39:24,855 - lava_callback - INFO - Received callback for node 69c5d09b2cb62322ad965f33 INFO: 172.20.0.1:40038 - "POST /node/69c5d09b2cb62322ad965f33 HTTP/1.0" 202 Accepted 2026-03-27 00:39:24,866 - lava_callback - INFO - Starting async job submit for node 69c5d09b2cb62322ad965f33 2026-03-27 00:39:24,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:39:24,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d09b2cb62322ad965f33 HTTP/1.1" 200 None 2026-03-27 00:39:24,960 - lava_callback - DEBUG - Processing node 69c5d09b2cb62322ad965f33: ltp-controllers 2026-03-27 00:39:25,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:39:26,281 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:39:26,283 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-controllers-69c5d09b2cb62322ad965f33/log.txt.gz INFO: 172.20.0.1:40040 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:39:26,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:39:27,421 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:39:27,423 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-controllers-69c5d09b2cb62322ad965f33/lava-logs.yaml 2026-03-27 00:39:27,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:39:27,828 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:39:27,831 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-controllers-69c5d09b2cb62322ad965f33/lava_callback.json.gz 2026-03-27 00:39:27,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:39:27,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d09b2cb62322ad965f33 HTTP/1.1" 200 None 2026-03-27 00:39:27,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:39:27,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-27 00:39:27,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:39:27,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d09b2cb62322ad965f33 HTTP/1.1" 200 None 2026-03-27 00:39:27,972 - lava_callback - INFO - Completed processing callback for node 69c5d09b2cb62322ad965f33 2026-03-27 00:39:29,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:39:29,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-27 00:39:33,609 - lava_callback - INFO - Received callback for node 69c5d09d2cb62322ad965f36 INFO: 172.20.0.1:49362 - "POST /node/69c5d09d2cb62322ad965f36 HTTP/1.0" 202 Accepted 2026-03-27 00:39:33,619 - lava_callback - INFO - Starting async job submit for node 69c5d09d2cb62322ad965f36 2026-03-27 00:39:33,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:39:33,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d09d2cb62322ad965f36 HTTP/1.1" 200 None 2026-03-27 00:39:33,707 - lava_callback - DEBUG - Processing node 69c5d09d2cb62322ad965f36: ltp-cve 2026-03-27 00:39:34,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:39:34,773 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:39:34,775 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cve-69c5d09d2cb62322ad965f36/log.txt.gz 2026-03-27 00:39:35,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:39:35,806 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:39:35,809 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cve-69c5d09d2cb62322ad965f36/lava-logs.yaml 2026-03-27 00:39:35,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:39:36,246 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:39:36,249 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cve-69c5d09d2cb62322ad965f36/lava_callback.json.gz 2026-03-27 00:39:36,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:39:36,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d09d2cb62322ad965f36 HTTP/1.1" 200 None 2026-03-27 00:39:36,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:39:36,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-27 00:39:36,376 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:39:36,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d09d2cb62322ad965f36 HTTP/1.1" 200 None 2026-03-27 00:39:36,449 - lava_callback - INFO - Completed processing callback for node 69c5d09d2cb62322ad965f36 INFO: 172.20.0.1:49374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59690 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:39:59,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:39:59,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:56548 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:40:08,189 - lava_callback - INFO - Received callback for node 69c5d07b2cb62322ad965ee4 INFO: 172.20.0.1:56550 - "POST /node/69c5d07b2cb62322ad965ee4 HTTP/1.0" 202 Accepted 2026-03-27 00:40:08,196 - lava_callback - INFO - Starting async job submit for node 69c5d07b2cb62322ad965ee4 2026-03-27 00:40:08,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:40:08,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d07b2cb62322ad965ee4 HTTP/1.1" 200 None 2026-03-27 00:40:08,262 - lava_callback - DEBUG - Processing node 69c5d07b2cb62322ad965ee4: baseline-arm-clabbe 2026-03-27 00:40:08,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:40:09,032 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:40:09,034 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c5d07b2cb62322ad965ee4/log.txt.gz 2026-03-27 00:40:09,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:40:09,784 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:40:09,787 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c5d07b2cb62322ad965ee4/lava-logs.yaml 2026-03-27 00:40:09,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:40:10,300 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:40:10,302 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c5d07b2cb62322ad965ee4/lava_callback.json.gz 2026-03-27 00:40:10,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:40:10,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d07b2cb62322ad965ee4 HTTP/1.1" 200 None 2026-03-27 00:40:10,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:40:10,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-27 00:40:10,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:40:10,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d07b2cb62322ad965ee4 HTTP/1.1" 200 None 2026-03-27 00:40:10,403 - lava_callback - INFO - Completed processing callback for node 69c5d07b2cb62322ad965ee4 INFO: 172.20.0.1:48056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48066 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45682 - "GET /node/69aff33dae557f2600f10239%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:45698 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:40:29,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:40:29,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:49534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57648 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:41:03,509 - lava_callback - INFO - Received callback for node 69c5b8a0abf66a8f4d9bcd1e INFO: 172.20.0.1:37886 - "POST /node/69c5b8a0abf66a8f4d9bcd1e HTTP/1.0" 202 Accepted 2026-03-27 00:41:03,516 - lava_callback - INFO - Starting async job submit for node 69c5b8a0abf66a8f4d9bcd1e 2026-03-27 00:41:03,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:41:03,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b8a0abf66a8f4d9bcd1e HTTP/1.1" 200 None 2026-03-27 00:41:03,560 - lava_callback - DEBUG - Processing node 69c5b8a0abf66a8f4d9bcd1e: baseline-arm64-broonie 2026-03-27 00:41:03,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:41:03,933 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:41:03,935 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c5b8a0abf66a8f4d9bcd1e/log.txt.gz 2026-03-27 00:41:04,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:41:04,267 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:41:04,269 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c5b8a0abf66a8f4d9bcd1e/lava-logs.yaml 2026-03-27 00:41:04,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:41:04,414 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:41:04,416 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c5b8a0abf66a8f4d9bcd1e/lava_callback.json.gz 2026-03-27 00:41:04,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:41:04,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b8a0abf66a8f4d9bcd1e HTTP/1.1" 200 None 2026-03-27 00:41:04,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:41:04,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-27 00:41:04,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:41:04,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5b8a0abf66a8f4d9bcd1e HTTP/1.1" 200 None 2026-03-27 00:41:04,503 - lava_callback - INFO - Completed processing callback for node 69c5b8a0abf66a8f4d9bcd1e INFO: 172.20.0.1:37894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48486 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53832 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:41:29,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:41:29,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:56292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56246 - "GET /node/67ff8032c0534c95c39deb21%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:43898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45526 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:42:14,444 - lava_callback - INFO - Received callback for node 69c5d24e2cb62322ad96609b INFO: 172.20.0.1:45532 - "POST /node/69c5d24e2cb62322ad96609b HTTP/1.0" 202 Accepted 2026-03-27 00:42:14,497 - lava_callback - INFO - Starting async job submit for node 69c5d24e2cb62322ad96609b 2026-03-27 00:42:14,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:42:14,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d24e2cb62322ad96609b HTTP/1.1" 200 None 2026-03-27 00:42:14,541 - lava_callback - DEBUG - Processing node 69c5d24e2cb62322ad96609b: baseline-arm 2026-03-27 00:42:14,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:42:14,833 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:42:14,835 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c5d24e2cb62322ad96609b/log.txt.gz 2026-03-27 00:42:14,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:42:15,068 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:42:15,069 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c5d24e2cb62322ad96609b/lava-logs.yaml 2026-03-27 00:42:15,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:45544 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-27 00:42:15,357 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:42:15,359 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c5d24e2cb62322ad96609b/lava_callback.json.gz 2026-03-27 00:42:15,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:42:15,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d24e2cb62322ad96609b HTTP/1.1" 200 None 2026-03-27 00:42:15,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:42:15,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-27 00:42:15,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:42:15,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d24e2cb62322ad96609b HTTP/1.1" 200 None 2026-03-27 00:42:15,469 - lava_callback - INFO - Completed processing callback for node 69c5d24e2cb62322ad96609b INFO: 172.20.0.1:45554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36850 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:42:29,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:42:29,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-27 00:42:32,696 - lava_callback - INFO - Received callback for node 69c5d2502cb62322ad96609e INFO: 172.20.0.1:54090 - "POST /node/69c5d2502cb62322ad96609e HTTP/1.0" 202 Accepted 2026-03-27 00:42:32,746 - lava_callback - INFO - Starting async job submit for node 69c5d2502cb62322ad96609e 2026-03-27 00:42:32,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:42:32,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d2502cb62322ad96609e HTTP/1.1" 200 None 2026-03-27 00:42:32,778 - lava_callback - DEBUG - Processing node 69c5d2502cb62322ad96609e: baseline-arm 2026-03-27 00:42:32,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:42:33,035 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:42:33,036 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c5d2502cb62322ad96609e/log.txt.gz 2026-03-27 00:42:33,112 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:42:33,289 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:42:33,290 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c5d2502cb62322ad96609e/lava-logs.yaml 2026-03-27 00:42:33,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:42:33,432 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:42:33,434 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c5d2502cb62322ad96609e/lava_callback.json.gz 2026-03-27 00:42:33,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:42:33,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d2502cb62322ad96609e HTTP/1.1" 200 None 2026-03-27 00:42:33,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:42:33,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-27 00:42:33,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:42:33,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d2502cb62322ad96609e HTTP/1.1" 200 None 2026-03-27 00:42:33,523 - lava_callback - INFO - Completed processing callback for node 69c5d2502cb62322ad96609e INFO: 172.20.0.1:54104 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:42:37,939 - lava_callback - INFO - Received callback for node 69c5d2742cb62322ad9660b1 INFO: 172.20.0.1:54106 - "POST /node/69c5d2742cb62322ad9660b1 HTTP/1.0" 202 Accepted 2026-03-27 00:42:37,944 - lava_callback - INFO - Starting async job submit for node 69c5d2742cb62322ad9660b1 2026-03-27 00:42:37,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:42:37,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d2742cb62322ad9660b1 HTTP/1.1" 200 None 2026-03-27 00:42:37,964 - lava_callback - DEBUG - Processing node 69c5d2742cb62322ad9660b1: baseline-arm 2026-03-27 00:42:37,972 - lava_callback - WARNING - Failed to upload log 2026-03-27 00:42:37,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:42:38,091 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:42:38,092 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c5d2742cb62322ad9660b1/lava-logs.yaml 2026-03-27 00:42:38,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:42:38,373 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:42:38,375 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c5d2742cb62322ad9660b1/lava_callback.json.gz 2026-03-27 00:42:38,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:42:38,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d2742cb62322ad9660b1 HTTP/1.1" 200 None 2026-03-27 00:42:38,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:42:38,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-27 00:42:38,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:42:38,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d2742cb62322ad9660b1 HTTP/1.1" 200 None 2026-03-27 00:42:38,442 - lava_callback - INFO - Completed processing callback for node 69c5d2742cb62322ad9660b1 2026-03-27 00:42:45,438 - lava_callback - INFO - Received callback for node 69c5d0a12cb62322ad965f3e INFO: 172.20.0.1:38816 - "POST /node/69c5d0a12cb62322ad965f3e HTTP/1.0" 202 Accepted 2026-03-27 00:42:45,445 - lava_callback - INFO - Starting async job submit for node 69c5d0a12cb62322ad965f3e 2026-03-27 00:42:45,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:42:45,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d0a12cb62322ad965f3e HTTP/1.1" 200 None 2026-03-27 00:42:45,485 - lava_callback - DEBUG - Processing node 69c5d0a12cb62322ad965f3e: ltp-fs_bind 2026-03-27 00:42:45,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:42:46,004 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:42:46,006 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69c5d0a12cb62322ad965f3e/log.txt.gz 2026-03-27 00:42:46,240 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:42:46,441 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:42:46,443 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69c5d0a12cb62322ad965f3e/lava-logs.yaml 2026-03-27 00:42:46,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:38832 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:42:46,592 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:42:46,593 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69c5d0a12cb62322ad965f3e/lava_callback.json.gz 2026-03-27 00:42:46,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:42:46,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d0a12cb62322ad965f3e HTTP/1.1" 200 None 2026-03-27 00:42:46,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:42:46,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-27 00:42:46,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:42:46,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d0a12cb62322ad965f3e HTTP/1.1" 200 None 2026-03-27 00:42:46,682 - lava_callback - INFO - Completed processing callback for node 69c5d0a12cb62322ad965f3e 2026-03-27 00:42:47,104 - lava_callback - INFO - Received callback for node 69c5d0a22cb62322ad965f41 INFO: 172.20.0.1:38838 - "POST /node/69c5d0a22cb62322ad965f41 HTTP/1.0" 202 Accepted 2026-03-27 00:42:47,111 - lava_callback - INFO - Starting async job submit for node 69c5d0a22cb62322ad965f41 2026-03-27 00:42:47,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:42:47,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d0a22cb62322ad965f41 HTTP/1.1" 200 None 2026-03-27 00:42:47,153 - lava_callback - DEBUG - Processing node 69c5d0a22cb62322ad965f41: ltp-fsx 2026-03-27 00:42:47,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:42:47,616 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:42:47,618 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fsx-69c5d0a22cb62322ad965f41/log.txt.gz 2026-03-27 00:42:47,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:42:48,017 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:42:48,019 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fsx-69c5d0a22cb62322ad965f41/lava-logs.yaml 2026-03-27 00:42:48,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:42:48,168 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:42:48,169 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fsx-69c5d0a22cb62322ad965f41/lava_callback.json.gz 2026-03-27 00:42:48,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:42:48,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d0a22cb62322ad965f41 HTTP/1.1" 200 None 2026-03-27 00:42:48,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:42:48,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-27 00:42:48,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:42:48,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d0a22cb62322ad965f41 HTTP/1.1" 200 None 2026-03-27 00:42:48,265 - lava_callback - INFO - Completed processing callback for node 69c5d0a22cb62322ad965f41 INFO: 172.20.0.1:44688 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:42:59,599 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:42:59,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.20.0.1:50394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45568 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37604 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:43:34,446 - lava_callback - INFO - Received callback for node 69c5d0a52cb62322ad965f47 INFO: 172.20.0.1:49200 - "POST /node/69c5d0a52cb62322ad965f47 HTTP/1.0" 202 Accepted 2026-03-27 00:43:34,452 - lava_callback - INFO - Starting async job submit for node 69c5d0a52cb62322ad965f47 2026-03-27 00:43:34,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:43:34,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d0a52cb62322ad965f47 HTTP/1.1" 200 None 2026-03-27 00:43:34,494 - lava_callback - DEBUG - Processing node 69c5d0a52cb62322ad965f47: ltp-ipc 2026-03-27 00:43:34,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:43:35,031 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:43:35,033 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-ipc-69c5d0a52cb62322ad965f47/log.txt.gz 2026-03-27 00:43:35,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:43:35,438 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:43:35,440 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-ipc-69c5d0a52cb62322ad965f47/lava-logs.yaml 2026-03-27 00:43:35,448 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:43:35,606 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:43:35,608 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-ipc-69c5d0a52cb62322ad965f47/lava_callback.json.gz 2026-03-27 00:43:35,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:43:35,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d0a52cb62322ad965f47 HTTP/1.1" 200 None 2026-03-27 00:43:35,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:43:35,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-27 00:43:35,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:43:35,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d0a52cb62322ad965f47 HTTP/1.1" 200 None 2026-03-27 00:43:35,702 - lava_callback - INFO - Completed processing callback for node 69c5d0a52cb62322ad965f47 INFO: 172.20.0.1:49208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35380 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:43:59,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:43:59,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-27 00:44:04,318 - lava_callback - INFO - Received callback for node 69c5d0a42cb62322ad965f44 INFO: 172.20.0.1:55830 - "POST /node/69c5d0a42cb62322ad965f44 HTTP/1.0" 202 Accepted 2026-03-27 00:44:04,325 - lava_callback - INFO - Starting async job submit for node 69c5d0a42cb62322ad965f44 2026-03-27 00:44:04,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:44:04,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d0a42cb62322ad965f44 HTTP/1.1" 200 None 2026-03-27 00:44:04,366 - lava_callback - DEBUG - Processing node 69c5d0a42cb62322ad965f44: ltp-input 2026-03-27 00:44:04,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:44:04,886 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:44:04,888 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-input-69c5d0a42cb62322ad965f44/log.txt.gz 2026-03-27 00:44:05,112 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:44:05,339 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:44:05,341 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-input-69c5d0a42cb62322ad965f44/lava-logs.yaml 2026-03-27 00:44:05,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:44:05,556 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:44:05,558 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-input-69c5d0a42cb62322ad965f44/lava_callback.json.gz 2026-03-27 00:44:05,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:44:05,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d0a42cb62322ad965f44 HTTP/1.1" 200 None 2026-03-27 00:44:05,599 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:44:05,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-27 00:44:05,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:44:05,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d0a42cb62322ad965f44 HTTP/1.1" 200 None 2026-03-27 00:44:05,653 - lava_callback - INFO - Completed processing callback for node 69c5d0a42cb62322ad965f44 INFO: 172.20.0.1:55842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52150 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60930 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:44:29,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:44:29,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:47114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47614 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47624 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:45:20,665 - lava_callback - INFO - Received callback for node 69c3300e18a3717aa65d2a3d INFO: 172.20.0.1:40346 - "POST /node/69c3300e18a3717aa65d2a3d HTTP/1.0" 202 Accepted 2026-03-27 00:45:20,671 - lava_callback - INFO - Starting async job submit for node 69c3300e18a3717aa65d2a3d 2026-03-27 00:45:20,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:45:20,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3300e18a3717aa65d2a3d HTTP/1.1" 200 None 2026-03-27 00:45:20,693 - lava_callback - DEBUG - Processing node 69c3300e18a3717aa65d2a3d: baseline-arm-clabbe 2026-03-27 00:45:20,693 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3300e18a3717aa65d2a3d 2026-03-27 00:45:20,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:45:20,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3300e18a3717aa65d2a3d HTTP/1.1" 200 None 2026-03-27 00:45:20,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:45:20,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-27 00:45:20,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:45:20,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3300e18a3717aa65d2a3d HTTP/1.1" 200 None 2026-03-27 00:45:20,748 - lava_callback - ERROR - Error processing callback for node 69c3300e18a3717aa65d2a3d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43282 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:45:49,891 - lava_callback - INFO - Received callback for node 69c5cff42cb62322ad965e07 INFO: 172.20.0.1:46966 - "POST /node/69c5cff42cb62322ad965e07 HTTP/1.0" 202 Accepted 2026-03-27 00:45:49,897 - lava_callback - INFO - Starting async job submit for node 69c5cff42cb62322ad965e07 2026-03-27 00:45:49,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:45:49,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cff42cb62322ad965e07 HTTP/1.1" 200 None 2026-03-27 00:45:49,964 - lava_callback - DEBUG - Processing node 69c5cff42cb62322ad965e07: baseline-x86 2026-03-27 00:45:50,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:45:50,525 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:45:50,526 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c5cff42cb62322ad965e07/log.txt.gz 2026-03-27 00:45:50,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:45:50,989 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:45:50,991 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c5cff42cb62322ad965e07/lava-logs.yaml 2026-03-27 00:45:51,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:45:51,191 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:45:51,193 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c5cff42cb62322ad965e07/lava_callback.json.gz 2026-03-27 00:45:51,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:45:51,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cff42cb62322ad965e07 HTTP/1.1" 200 None 2026-03-27 00:45:51,240 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:45:51,254 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc5a2cb62322ad965dce HTTP/1.1" 200 None 2026-03-27 00:45:51,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:45:51,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5cff42cb62322ad965e07 HTTP/1.1" 200 None 2026-03-27 00:45:51,286 - lava_callback - INFO - Completed processing callback for node 69c5cff42cb62322ad965e07 INFO: 172.20.0.1:46976 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:45:59,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:45:59,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:50948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42068 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34542 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:46:40,656 - lava_callback - INFO - Received callback for node 69c3305a18a3717aa65d2a53 INFO: 172.20.0.1:39322 - "POST /node/69c3305a18a3717aa65d2a53 HTTP/1.0" 202 Accepted 2026-03-27 00:46:40,661 - lava_callback - INFO - Starting async job submit for node 69c3305a18a3717aa65d2a53 2026-03-27 00:46:40,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:46:40,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3305a18a3717aa65d2a53 HTTP/1.1" 200 None 2026-03-27 00:46:40,677 - lava_callback - DEBUG - Processing node 69c3305a18a3717aa65d2a53: baseline-arm-clabbe 2026-03-27 00:46:40,677 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3305a18a3717aa65d2a53 2026-03-27 00:46:40,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:46:40,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3305a18a3717aa65d2a53 HTTP/1.1" 200 None 2026-03-27 00:46:40,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:46:40,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-27 00:46:40,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:46:40,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3305a18a3717aa65d2a53 HTTP/1.1" 200 None 2026-03-27 00:46:40,734 - lava_callback - ERROR - Error processing callback for node 69c3305a18a3717aa65d2a53: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-27 00:46:44,115 - lava_callback - INFO - Received callback for node 69c5d0a72cb62322ad965f4e INFO: 172.20.0.1:39334 - "POST /node/69c5d0a72cb62322ad965f4e HTTP/1.0" 202 Accepted 2026-03-27 00:46:44,122 - lava_callback - INFO - Starting async job submit for node 69c5d0a72cb62322ad965f4e 2026-03-27 00:46:44,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:46:44,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d0a72cb62322ad965f4e HTTP/1.1" 200 None 2026-03-27 00:46:44,164 - lava_callback - DEBUG - Processing node 69c5d0a72cb62322ad965f4e: ltp-pty 2026-03-27 00:46:44,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:46:44,644 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:46:44,646 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-pty-69c5d0a72cb62322ad965f4e/log.txt.gz 2026-03-27 00:46:44,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:46:45,037 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:46:45,039 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-pty-69c5d0a72cb62322ad965f4e/lava-logs.yaml 2026-03-27 00:46:45,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:46:45,199 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:46:45,201 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-pty-69c5d0a72cb62322ad965f4e/lava_callback.json.gz 2026-03-27 00:46:45,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:46:45,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d0a72cb62322ad965f4e HTTP/1.1" 200 None 2026-03-27 00:46:45,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:46:45,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-27 00:46:45,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:46:45,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d0a72cb62322ad965f4e HTTP/1.1" 200 None 2026-03-27 00:46:45,282 - lava_callback - INFO - Completed processing callback for node 69c5d0a72cb62322ad965f4e 2026-03-27 00:46:45,740 - lava_callback - INFO - Received callback for node 69c5d0a82cb62322ad965f51 INFO: 172.20.0.1:39350 - "POST /node/69c5d0a82cb62322ad965f51 HTTP/1.0" 202 Accepted 2026-03-27 00:46:45,746 - lava_callback - INFO - Starting async job submit for node 69c5d0a82cb62322ad965f51 2026-03-27 00:46:45,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:46:45,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d0a82cb62322ad965f51 HTTP/1.1" 200 None 2026-03-27 00:46:45,805 - lava_callback - DEBUG - Processing node 69c5d0a82cb62322ad965f51: ltp-sched 2026-03-27 00:46:46,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:46:46,275 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:46:46,277 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-sched-69c5d0a82cb62322ad965f51/log.txt.gz 2026-03-27 00:46:46,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:39362 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:46:46,697 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:46:46,699 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-sched-69c5d0a82cb62322ad965f51/lava-logs.yaml 2026-03-27 00:46:46,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:46:46,855 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:46:46,856 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-sched-69c5d0a82cb62322ad965f51/lava_callback.json.gz 2026-03-27 00:46:46,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:46:46,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d0a82cb62322ad965f51 HTTP/1.1" 200 None 2026-03-27 00:46:46,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:46:46,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-27 00:46:46,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:46:46,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d0a82cb62322ad965f51 HTTP/1.1" 200 None 2026-03-27 00:46:46,978 - lava_callback - INFO - Completed processing callback for node 69c5d0a82cb62322ad965f51 INFO: 172.20.0.1:45878 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:46:59,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:46:59,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:57834 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:47:07,627 - lava_callback - INFO - Received callback for node 69c3308118a3717aa65d2a60 INFO: 172.20.0.1:57844 - "POST /node/69c3308118a3717aa65d2a60 HTTP/1.0" 202 Accepted 2026-03-27 00:47:07,633 - lava_callback - INFO - Starting async job submit for node 69c3308118a3717aa65d2a60 2026-03-27 00:47:07,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:47:07,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3308118a3717aa65d2a60 HTTP/1.1" 200 None 2026-03-27 00:47:07,648 - lava_callback - DEBUG - Processing node 69c3308118a3717aa65d2a60: baseline-arm-baylibre 2026-03-27 00:47:07,649 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3308118a3717aa65d2a60 2026-03-27 00:47:07,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:47:07,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3308118a3717aa65d2a60 HTTP/1.1" 200 None 2026-03-27 00:47:07,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:47:07,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-27 00:47:07,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:47:07,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3308118a3717aa65d2a60 HTTP/1.1" 200 None 2026-03-27 00:47:07,705 - lava_callback - ERROR - Error processing callback for node 69c3308118a3717aa65d2a60: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55082 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38062 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:47:35,901 - lava_callback - INFO - Received callback for node 69c5d0aa2cb62322ad965f54 INFO: 172.20.0.1:42702 - "POST /node/69c5d0aa2cb62322ad965f54 HTTP/1.0" 202 Accepted 2026-03-27 00:47:35,910 - lava_callback - INFO - Starting async job submit for node 69c5d0aa2cb62322ad965f54 2026-03-27 00:47:35,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:47:35,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d0aa2cb62322ad965f54 HTTP/1.1" 200 None 2026-03-27 00:47:35,995 - lava_callback - DEBUG - Processing node 69c5d0aa2cb62322ad965f54: ltp-smoke INFO: 172.20.0.1:42716 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:47:36,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:47:37,550 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:47:37,552 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c5d0aa2cb62322ad965f54/log.txt.gz 2026-03-27 00:47:37,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:47:39,411 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:47:39,414 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c5d0aa2cb62322ad965f54/lava-logs.yaml 2026-03-27 00:47:39,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:47:40,458 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:47:40,460 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c5d0aa2cb62322ad965f54/lava_callback.json.gz 2026-03-27 00:47:40,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:47:40,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d0aa2cb62322ad965f54 HTTP/1.1" 200 None 2026-03-27 00:47:40,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:47:40,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-27 00:47:40,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:47:40,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d0aa2cb62322ad965f54 HTTP/1.1" 200 None 2026-03-27 00:47:40,600 - lava_callback - INFO - Completed processing callback for node 69c5d0aa2cb62322ad965f54 INFO: 172.20.0.1:37626 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:47:47,612 - lava_callback - INFO - Received callback for node 69c330a818a3717aa65d2a6a INFO: 172.20.0.1:37642 - "POST /node/69c330a818a3717aa65d2a6a HTTP/1.0" 202 Accepted 2026-03-27 00:47:47,620 - lava_callback - INFO - Starting async job submit for node 69c330a818a3717aa65d2a6a 2026-03-27 00:47:47,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:47:47,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c330a818a3717aa65d2a6a HTTP/1.1" 200 None 2026-03-27 00:47:47,656 - lava_callback - DEBUG - Processing node 69c330a818a3717aa65d2a6a: baseline-arm-baylibre 2026-03-27 00:47:47,656 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c330a818a3717aa65d2a6a 2026-03-27 00:47:47,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:47:47,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c330a818a3717aa65d2a6a HTTP/1.1" 200 None 2026-03-27 00:47:47,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:47:47,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-27 00:47:47,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:47:47,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c330a818a3717aa65d2a6a HTTP/1.1" 200 None 2026-03-27 00:47:47,753 - lava_callback - ERROR - Error processing callback for node 69c330a818a3717aa65d2a6a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-27 00:47:55,705 - lava_callback - INFO - Received callback for node 69c5d0582cb62322ad965e5d INFO: 172.20.0.1:47210 - "POST /node/69c5d0582cb62322ad965e5d HTTP/1.0" 202 Accepted 2026-03-27 00:47:55,715 - lava_callback - INFO - Starting async job submit for node 69c5d0582cb62322ad965e5d 2026-03-27 00:47:55,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:47:55,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d0582cb62322ad965e5d HTTP/1.1" 200 None 2026-03-27 00:47:55,835 - lava_callback - DEBUG - Processing node 69c5d0582cb62322ad965e5d: baseline-arm INFO: 172.20.0.1:47226 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:47:56,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:47:57,460 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:47:57,462 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c5d0582cb62322ad965e5d/log.txt.gz 2026-03-27 00:47:57,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:47:59,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:47:59,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-27 00:47:59,801 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:47:59,804 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c5d0582cb62322ad965e5d/lava-logs.yaml 2026-03-27 00:47:59,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:48:01,112 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:48:01,114 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c5d0582cb62322ad965e5d/lava_callback.json.gz DEBUG: Setup failed for 69c5d0582cb62322ad965e5d. Transitting job node result: incomplete -> fail 2026-03-27 00:48:01,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:48:01,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d0582cb62322ad965e5d HTTP/1.1" 200 None 2026-03-27 00:48:01,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:48:01,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-27 00:48:01,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:48:01,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d0582cb62322ad965e5d HTTP/1.1" 200 None 2026-03-27 00:48:01,570 - lava_callback - INFO - Completed processing callback for node 69c5d0582cb62322ad965e5d INFO: 172.20.0.1:55702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50294 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54814 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:48:27,675 - lava_callback - INFO - Received callback for node 69c330d018a3717aa65d2a74 INFO: 172.20.0.1:54830 - "POST /node/69c330d018a3717aa65d2a74 HTTP/1.0" 202 Accepted 2026-03-27 00:48:27,684 - lava_callback - INFO - Starting async job submit for node 69c330d018a3717aa65d2a74 2026-03-27 00:48:27,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:48:27,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c330d018a3717aa65d2a74 HTTP/1.1" 200 None 2026-03-27 00:48:27,711 - lava_callback - DEBUG - Processing node 69c330d018a3717aa65d2a74: baseline-arm-baylibre 2026-03-27 00:48:27,712 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c330d018a3717aa65d2a74 2026-03-27 00:48:27,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:48:27,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c330d018a3717aa65d2a74 HTTP/1.1" 200 None 2026-03-27 00:48:27,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:48:27,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-27 00:48:27,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:48:27,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c330d018a3717aa65d2a74 HTTP/1.1" 200 None 2026-03-27 00:48:27,808 - lava_callback - ERROR - Error processing callback for node 69c330d018a3717aa65d2a74: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-27 00:48:29,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:48:29,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:34756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48994 - "GET /node/6990aa04c45febfe4452167d%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:49002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43882 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43892 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:49:20,807 - lava_callback - INFO - Received callback for node 69c330fb18a3717aa65d2a7f INFO: 172.20.0.1:47460 - "POST /node/69c330fb18a3717aa65d2a7f HTTP/1.0" 202 Accepted 2026-03-27 00:49:20,816 - lava_callback - INFO - Starting async job submit for node 69c330fb18a3717aa65d2a7f 2026-03-27 00:49:20,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:49:20,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c330fb18a3717aa65d2a7f HTTP/1.1" 200 None 2026-03-27 00:49:20,837 - lava_callback - DEBUG - Processing node 69c330fb18a3717aa65d2a7f: baseline-arm-clabbe 2026-03-27 00:49:20,837 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c330fb18a3717aa65d2a7f 2026-03-27 00:49:20,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:49:20,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c330fb18a3717aa65d2a7f HTTP/1.1" 200 None 2026-03-27 00:49:20,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:49:20,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-27 00:49:20,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:49:20,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c330fb18a3717aa65d2a7f HTTP/1.1" 200 None 2026-03-27 00:49:20,921 - lava_callback - ERROR - Error processing callback for node 69c330fb18a3717aa65d2a7f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44428 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:49:47,734 - lava_callback - INFO - Received callback for node 69c3311e18a3717aa65d2a8a INFO: 172.20.0.1:44436 - "POST /node/69c3311e18a3717aa65d2a8a HTTP/1.0" 202 Accepted 2026-03-27 00:49:47,744 - lava_callback - INFO - Starting async job submit for node 69c3311e18a3717aa65d2a8a 2026-03-27 00:49:47,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:49:47,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3311e18a3717aa65d2a8a HTTP/1.1" 200 None 2026-03-27 00:49:47,769 - lava_callback - DEBUG - Processing node 69c3311e18a3717aa65d2a8a: baseline-arm-baylibre 2026-03-27 00:49:47,769 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3311e18a3717aa65d2a8a 2026-03-27 00:49:47,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:49:47,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3311e18a3717aa65d2a8a HTTP/1.1" 200 None 2026-03-27 00:49:47,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:49:47,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-27 00:49:47,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:49:47,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3311e18a3717aa65d2a8a HTTP/1.1" 200 None 2026-03-27 00:49:47,849 - lava_callback - ERROR - Error processing callback for node 69c3311e18a3717aa65d2a8a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41880 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41894 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:50:25,008 - lava_callback - INFO - Received callback for node 69c5d43f2cb62322ad966196 INFO: 172.20.0.1:52784 - "POST /node/69c5d43f2cb62322ad966196 HTTP/1.0" 202 Accepted 2026-03-27 00:50:25,019 - lava_callback - INFO - Starting async job submit for node 69c5d43f2cb62322ad966196 2026-03-27 00:50:25,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:50:25,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d43f2cb62322ad966196 HTTP/1.1" 200 None 2026-03-27 00:50:25,048 - lava_callback - DEBUG - Processing node 69c5d43f2cb62322ad966196: baseline-arm 2026-03-27 00:50:25,056 - lava_callback - WARNING - Failed to upload log 2026-03-27 00:50:25,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:52794 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:50:27,443 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:50:27,445 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c5d43f2cb62322ad966196/lava-logs.yaml 2026-03-27 00:50:27,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:50:27,749 - lava_callback - INFO - Received callback for node 69c3314718a3717aa65d2a95 INFO: 172.20.0.1:52798 - "POST /node/69c3314718a3717aa65d2a95 HTTP/1.0" 202 Accepted 2026-03-27 00:50:27,758 - lava_callback - INFO - Starting async job submit for node 69c3314718a3717aa65d2a95 2026-03-27 00:50:27,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:50:27,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3314718a3717aa65d2a95 HTTP/1.1" 200 None 2026-03-27 00:50:27,782 - lava_callback - DEBUG - Processing node 69c3314718a3717aa65d2a95: baseline-arm-baylibre 2026-03-27 00:50:27,782 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3314718a3717aa65d2a95 2026-03-27 00:50:27,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:50:27,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3314718a3717aa65d2a95 HTTP/1.1" 200 None 2026-03-27 00:50:27,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:50:27,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-27 00:50:27,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:50:27,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3314718a3717aa65d2a95 HTTP/1.1" 200 None 2026-03-27 00:50:27,865 - lava_callback - ERROR - Error processing callback for node 69c3314718a3717aa65d2a95: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-27 00:50:30,016 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:50:30,018 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c5d43f2cb62322ad966196/lava_callback.json.gz 2026-03-27 00:50:30,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:50:30,044 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d43f2cb62322ad966196 HTTP/1.1" 200 None 2026-03-27 00:50:30,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:50:30,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-27 00:50:30,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:50:30,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d43f2cb62322ad966196 HTTP/1.1" 200 None 2026-03-27 00:50:30,113 - lava_callback - INFO - Completed processing callback for node 69c5d43f2cb62322ad966196 INFO: 172.20.0.1:57956 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:50:44,136 - lava_callback - INFO - Received callback for node 69c5d0ad2cb62322ad965f5a INFO: 172.20.0.1:32946 - "POST /node/69c5d0ad2cb62322ad965f5a HTTP/1.0" 202 Accepted 2026-03-27 00:50:44,146 - lava_callback - INFO - Starting async job submit for node 69c5d0ad2cb62322ad965f5a 2026-03-27 00:50:44,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:50:44,203 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d0ad2cb62322ad965f5a HTTP/1.1" 200 None 2026-03-27 00:50:44,204 - lava_callback - DEBUG - Processing node 69c5d0ad2cb62322ad965f5a: ltp-syscalls-ipc 2026-03-27 00:50:44,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:50:45,765 - lava_callback - INFO - Received callback for node 69c5d0af2cb62322ad965f61 INFO: 172.20.0.1:32962 - "POST /node/69c5d0af2cb62322ad965f61 HTTP/1.0" 202 Accepted 2026-03-27 00:50:45,774 - lava_callback - INFO - Starting async job submit for node 69c5d0af2cb62322ad965f61 2026-03-27 00:50:45,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:50:45,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d0af2cb62322ad965f61 HTTP/1.1" 200 None 2026-03-27 00:50:45,825 - lava_callback - DEBUG - Processing node 69c5d0af2cb62322ad965f61: ltp-timers 2026-03-27 00:50:46,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:32964 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:50:47,645 - lava_callback - INFO - Received callback for node 69c5d0b02cb62322ad965f64 2026-03-27 00:50:47,656 - lava_callback - INFO - Starting async job submit for node 69c5d0b02cb62322ad965f64 INFO: 172.20.0.1:32978 - "POST /node/69c5d0b02cb62322ad965f64 HTTP/1.0" 202 Accepted 2026-03-27 00:50:47,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:50:47,715 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:50:47,717 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69c5d0ad2cb62322ad965f5a/log.txt.gz 2026-03-27 00:50:47,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d0b02cb62322ad965f64 HTTP/1.1" 200 None 2026-03-27 00:50:47,731 - lava_callback - DEBUG - Processing node 69c5d0b02cb62322ad965f64: ltp-watchqueue 2026-03-27 00:50:48,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:50:48,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:50:49,456 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:50:49,458 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-timers-69c5d0af2cb62322ad965f61/log.txt.gz 2026-03-27 00:50:49,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:50:51,510 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:50:51,513 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69c5d0ad2cb62322ad965f5a/lava-logs.yaml 2026-03-27 00:50:51,523 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:50:51,525 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69c5d0b02cb62322ad965f64/log.txt.gz 2026-03-27 00:50:51,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:50:51,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:50:52,910 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:50:52,912 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-timers-69c5d0af2cb62322ad965f61/lava-logs.yaml 2026-03-27 00:50:52,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:50:54,177 - lava_callback - INFO - Received callback for node 69c5d0462cb62322ad965e3c 2026-03-27 00:50:54,186 - lava_callback - INFO - Starting async job submit for node 69c5d0462cb62322ad965e3c INFO: 172.20.0.1:41108 - "POST /node/69c5d0462cb62322ad965e3c HTTP/1.0" 202 Accepted 2026-03-27 00:50:54,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:50:54,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d0462cb62322ad965e3c HTTP/1.1" 200 None 2026-03-27 00:50:54,324 - lava_callback - DEBUG - Processing node 69c5d0462cb62322ad965e3c: ltp-smoke 2026-03-27 00:50:54,658 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:50:54,696 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69c5d0ad2cb62322ad965f5a/lava_callback.json.gz 2026-03-27 00:50:54,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:50:54,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d0ad2cb62322ad965f5a HTTP/1.1" 200 None 2026-03-27 00:50:54,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:50:54,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:50:54,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-27 00:50:54,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:50:54,993 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:50:54,996 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69c5d0b02cb62322ad965f64/lava-logs.yaml 2026-03-27 00:50:55,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:50:55,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d0ad2cb62322ad965f5a HTTP/1.1" 200 None 2026-03-27 00:50:55,042 - lava_callback - INFO - Completed processing callback for node 69c5d0ad2cb62322ad965f5a 2026-03-27 00:50:55,995 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:50:55,997 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-timers-69c5d0af2cb62322ad965f61/lava_callback.json.gz 2026-03-27 00:50:56,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:50:56,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d0af2cb62322ad965f61 HTTP/1.1" 200 None 2026-03-27 00:50:56,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:50:56,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-27 00:50:56,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:50:56,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d0af2cb62322ad965f61 HTTP/1.1" 200 None 2026-03-27 00:50:56,130 - lava_callback - INFO - Completed processing callback for node 69c5d0af2cb62322ad965f61 INFO: 172.20.0.1:41122 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:50:58,013 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:50:58,016 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c5d0462cb62322ad965e3c/log.txt.gz 2026-03-27 00:50:58,174 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:50:58,184 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69c5d0b02cb62322ad965f64/lava_callback.json.gz 2026-03-27 00:50:58,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:50:58,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d0b02cb62322ad965f64 HTTP/1.1" 200 None 2026-03-27 00:50:58,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:50:58,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-27 00:50:58,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:50:58,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:50:58,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d0b02cb62322ad965f64 HTTP/1.1" 200 None 2026-03-27 00:50:58,493 - lava_callback - INFO - Completed processing callback for node 69c5d0b02cb62322ad965f64 2026-03-27 00:50:59,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:50:59,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-27 00:51:01,612 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:51:01,614 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c5d0462cb62322ad965e3c/lava-logs.yaml 2026-03-27 00:51:01,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:51:04,664 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:51:04,666 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c5d0462cb62322ad965e3c/lava_callback.json.gz 2026-03-27 00:51:04,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:51:04,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d0462cb62322ad965e3c HTTP/1.1" 200 None 2026-03-27 00:51:04,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:51:04,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc312cb62322ad965dad HTTP/1.1" 200 None 2026-03-27 00:51:04,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:51:04,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d0462cb62322ad965e3c HTTP/1.1" 200 None 2026-03-27 00:51:05,092 - lava_callback - INFO - Completed processing callback for node 69c5d0462cb62322ad965e3c INFO: 172.20.0.1:58650 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:51:07,845 - lava_callback - INFO - Received callback for node 69c3316f18a3717aa65d2aa1 INFO: 172.20.0.1:58666 - "POST /node/69c3316f18a3717aa65d2aa1 HTTP/1.0" 202 Accepted 2026-03-27 00:51:07,853 - lava_callback - INFO - Starting async job submit for node 69c3316f18a3717aa65d2aa1 2026-03-27 00:51:07,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:51:07,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3316f18a3717aa65d2aa1 HTTP/1.1" 200 None 2026-03-27 00:51:07,880 - lava_callback - DEBUG - Processing node 69c3316f18a3717aa65d2aa1: baseline-arm-baylibre 2026-03-27 00:51:07,881 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3316f18a3717aa65d2aa1 2026-03-27 00:51:07,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:51:07,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3316f18a3717aa65d2aa1 HTTP/1.1" 200 None 2026-03-27 00:51:07,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:51:07,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-27 00:51:07,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:51:07,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3316f18a3717aa65d2aa1 HTTP/1.1" 200 None 2026-03-27 00:51:07,971 - lava_callback - ERROR - Error processing callback for node 69c3316f18a3717aa65d2aa1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54808 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34454 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:51:29,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:51:29,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:50348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50880 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:52:00,869 - lava_callback - INFO - Received callback for node 69c3319a18a3717aa65d2ab2 INFO: 172.20.0.1:57772 - "POST /node/69c3319a18a3717aa65d2ab2 HTTP/1.0" 202 Accepted 2026-03-27 00:52:00,877 - lava_callback - INFO - Starting async job submit for node 69c3319a18a3717aa65d2ab2 2026-03-27 00:52:00,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:52:00,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3319a18a3717aa65d2ab2 HTTP/1.1" 200 None 2026-03-27 00:52:00,896 - lava_callback - DEBUG - Processing node 69c3319a18a3717aa65d2ab2: baseline-arm-clabbe 2026-03-27 00:52:00,896 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3319a18a3717aa65d2ab2 2026-03-27 00:52:00,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:52:00,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3319a18a3717aa65d2ab2 HTTP/1.1" 200 None 2026-03-27 00:52:00,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:52:00,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-27 00:52:00,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:52:00,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3319a18a3717aa65d2ab2 HTTP/1.1" 200 None 2026-03-27 00:52:00,984 - lava_callback - ERROR - Error processing callback for node 69c3319a18a3717aa65d2ab2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38712 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38722 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:52:20,864 - lava_callback - INFO - Received callback for node 69c331c118a3717aa65d2ac4 INFO: 172.20.0.1:41716 - "POST /node/69c331c118a3717aa65d2ac4 HTTP/1.0" 202 Accepted 2026-03-27 00:52:20,872 - lava_callback - INFO - Starting async job submit for node 69c331c118a3717aa65d2ac4 2026-03-27 00:52:20,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:52:20,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c331c118a3717aa65d2ac4 HTTP/1.1" 200 None 2026-03-27 00:52:20,910 - lava_callback - DEBUG - Processing node 69c331c118a3717aa65d2ac4: baseline-arm-clabbe 2026-03-27 00:52:20,910 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c331c118a3717aa65d2ac4 2026-03-27 00:52:20,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:52:20,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c331c118a3717aa65d2ac4 HTTP/1.1" 200 None 2026-03-27 00:52:20,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:52:20,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-27 00:52:20,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:52:20,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c331c118a3717aa65d2ac4 HTTP/1.1" 200 None 2026-03-27 00:52:20,989 - lava_callback - ERROR - Error processing callback for node 69c331c118a3717aa65d2ac4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59804 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:53:00,920 - lava_callback - INFO - Received callback for node 69c331e818a3717aa65d2acb INFO: 172.20.0.1:51454 - "POST /node/69c331e818a3717aa65d2acb HTTP/1.0" 202 Accepted 2026-03-27 00:53:00,927 - lava_callback - INFO - Starting async job submit for node 69c331e818a3717aa65d2acb 2026-03-27 00:53:00,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:53:00,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c331e818a3717aa65d2acb HTTP/1.1" 200 None 2026-03-27 00:53:00,956 - lava_callback - DEBUG - Processing node 69c331e818a3717aa65d2acb: baseline-arm-clabbe 2026-03-27 00:53:00,956 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c331e818a3717aa65d2acb 2026-03-27 00:53:00,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:53:00,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c331e818a3717aa65d2acb HTTP/1.1" 200 None 2026-03-27 00:53:00,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:53:01,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-27 00:53:01,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:53:01,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c331e818a3717aa65d2acb HTTP/1.1" 200 None 2026-03-27 00:53:01,068 - lava_callback - ERROR - Error processing callback for node 69c331e818a3717aa65d2acb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54592 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56440 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:53:40,953 - lava_callback - INFO - Received callback for node 69c3320e18a3717aa65d2ad9 INFO: 172.20.0.1:59580 - "POST /node/69c3320e18a3717aa65d2ad9 HTTP/1.0" 202 Accepted 2026-03-27 00:53:40,961 - lava_callback - INFO - Starting async job submit for node 69c3320e18a3717aa65d2ad9 2026-03-27 00:53:40,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:53:40,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3320e18a3717aa65d2ad9 HTTP/1.1" 200 None 2026-03-27 00:53:40,994 - lava_callback - DEBUG - Processing node 69c3320e18a3717aa65d2ad9: baseline-arm-clabbe 2026-03-27 00:53:40,995 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3320e18a3717aa65d2ad9 2026-03-27 00:53:40,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:53:41,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3320e18a3717aa65d2ad9 HTTP/1.1" 200 None 2026-03-27 00:53:41,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:53:41,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-27 00:53:41,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:53:41,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3320e18a3717aa65d2ad9 HTTP/1.1" 200 None 2026-03-27 00:53:41,090 - lava_callback - ERROR - Error processing callback for node 69c3320e18a3717aa65d2ad9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-27 00:53:44,253 - lava_callback - INFO - Received callback for node 69c5d0852cb62322ad965efc INFO: 172.20.0.1:59586 - "POST /node/69c5d0852cb62322ad965efc HTTP/1.0" 202 Accepted 2026-03-27 00:53:44,302 - lava_callback - INFO - Starting async job submit for node 69c5d0852cb62322ad965efc 2026-03-27 00:53:44,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:53:44,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d0852cb62322ad965efc HTTP/1.1" 200 None 2026-03-27 00:53:44,367 - lava_callback - DEBUG - Processing node 69c5d0852cb62322ad965efc: baseline-arm-pengutronix 2026-03-27 00:53:44,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:59600 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:53:47,361 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:53:47,363 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c5d0852cb62322ad965efc/log.txt.gz 2026-03-27 00:53:47,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:53:50,283 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:53:50,286 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c5d0852cb62322ad965efc/lava-logs.yaml 2026-03-27 00:53:50,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:53:52,892 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:53:52,894 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c5d0852cb62322ad965efc/lava_callback.json.gz DEBUG: Setup failed for 69c5d0852cb62322ad965efc. Transitting job node result: incomplete -> fail 2026-03-27 00:53:52,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:53:52,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d0852cb62322ad965efc HTTP/1.1" 200 None 2026-03-27 00:53:53,001 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:53:53,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-27 00:53:53,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:53:53,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d0852cb62322ad965efc HTTP/1.1" 200 None 2026-03-27 00:53:53,127 - lava_callback - INFO - Completed processing callback for node 69c5d0852cb62322ad965efc INFO: 172.20.0.1:42948 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:53:59,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:53:59,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:35286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49392 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49396 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:54:21,016 - lava_callback - INFO - Received callback for node 69c3323518a3717aa65d2aea INFO: 172.20.0.1:43760 - "POST /node/69c3323518a3717aa65d2aea HTTP/1.0" 202 Accepted 2026-03-27 00:54:21,025 - lava_callback - INFO - Starting async job submit for node 69c3323518a3717aa65d2aea 2026-03-27 00:54:21,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:54:21,063 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3323518a3717aa65d2aea HTTP/1.1" 200 None 2026-03-27 00:54:21,064 - lava_callback - DEBUG - Processing node 69c3323518a3717aa65d2aea: baseline-arm-clabbe 2026-03-27 00:54:21,065 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3323518a3717aa65d2aea 2026-03-27 00:54:21,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:54:21,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3323518a3717aa65d2aea HTTP/1.1" 200 None 2026-03-27 00:54:21,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:54:21,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-27 00:54:21,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:54:21,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3323518a3717aa65d2aea HTTP/1.1" 200 None 2026-03-27 00:54:21,141 - lava_callback - ERROR - Error processing callback for node 69c3323518a3717aa65d2aea: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34494 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:54:37,790 - lava_callback - INFO - Received callback for node 69c5d5302cb62322ad96621e INFO: 172.20.0.1:34496 - "POST /node/69c5d5302cb62322ad96621e HTTP/1.0" 202 Accepted 2026-03-27 00:54:37,798 - lava_callback - INFO - Starting async job submit for node 69c5d5302cb62322ad96621e 2026-03-27 00:54:37,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:54:37,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5302cb62322ad96621e HTTP/1.1" 200 None 2026-03-27 00:54:37,855 - lava_callback - DEBUG - Processing node 69c5d5302cb62322ad96621e: baseline-arm64 2026-03-27 00:54:38,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:54:40,245 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:54:40,247 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5d5302cb62322ad96621e/log.txt.gz 2026-03-27 00:54:40,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:54:42,638 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:54:42,640 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5d5302cb62322ad96621e/lava-logs.yaml 2026-03-27 00:54:42,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:54:44,734 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:54:44,736 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5d5302cb62322ad96621e/lava_callback.json.gz 2026-03-27 00:54:44,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:54:44,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5302cb62322ad96621e HTTP/1.1" 200 None 2026-03-27 00:54:44,779 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:54:44,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-27 00:54:44,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:54:44,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d5302cb62322ad96621e HTTP/1.1" 200 None 2026-03-27 00:54:44,842 - lava_callback - INFO - Completed processing callback for node 69c5d5302cb62322ad96621e INFO: 172.20.0.1:37152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51348 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:54:59,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:54:59,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:58622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41820 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41836 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:55:22,881 - lava_callback - INFO - Received callback for node 69c5d5502cb62322ad966253 INFO: 172.20.0.1:48824 - "POST /node/69c5d5502cb62322ad966253 HTTP/1.0" 202 Accepted 2026-03-27 00:55:22,891 - lava_callback - INFO - Starting async job submit for node 69c5d5502cb62322ad966253 2026-03-27 00:55:22,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:55:22,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5502cb62322ad966253 HTTP/1.1" 200 None 2026-03-27 00:55:22,944 - lava_callback - DEBUG - Processing node 69c5d5502cb62322ad966253: baseline-arm64 2026-03-27 00:55:23,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:55:24,713 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:55:24,716 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5d5502cb62322ad966253/log.txt.gz 2026-03-27 00:55:24,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:55:26,427 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:55:26,429 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5d5502cb62322ad966253/lava-logs.yaml 2026-03-27 00:55:26,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:48836 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:55:28,157 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:55:28,160 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5d5502cb62322ad966253/lava_callback.json.gz 2026-03-27 00:55:28,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:55:28,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5502cb62322ad966253 HTTP/1.1" 200 None 2026-03-27 00:55:28,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:55:28,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-27 00:55:28,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:55:28,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d5502cb62322ad966253 HTTP/1.1" 200 None 2026-03-27 00:55:28,273 - lava_callback - INFO - Completed processing callback for node 69c5d5502cb62322ad966253 2026-03-27 00:55:29,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:55:29,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-27 00:55:31,900 - lava_callback - INFO - Received callback for node 69c5d5422cb62322ad966237 INFO: 172.20.0.1:56736 - "POST /node/69c5d5422cb62322ad966237 HTTP/1.0" 202 Accepted 2026-03-27 00:55:31,907 - lava_callback - INFO - Starting async job submit for node 69c5d5422cb62322ad966237 2026-03-27 00:55:31,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:55:31,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5422cb62322ad966237 HTTP/1.1" 200 None 2026-03-27 00:55:31,968 - lava_callback - DEBUG - Processing node 69c5d5422cb62322ad966237: baseline-arm64 2026-03-27 00:55:32,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:55:32,540 - lava_callback - INFO - Received callback for node 69c5d5332cb62322ad966224 INFO: 172.20.0.1:56746 - "POST /node/69c5d5332cb62322ad966224 HTTP/1.0" 202 Accepted 2026-03-27 00:55:32,574 - lava_callback - INFO - Starting async job submit for node 69c5d5332cb62322ad966224 2026-03-27 00:55:32,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:55:32,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5332cb62322ad966224 HTTP/1.1" 200 None 2026-03-27 00:55:32,634 - lava_callback - DEBUG - Processing node 69c5d5332cb62322ad966224: baseline-arm64 2026-03-27 00:55:33,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:55:33,964 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:55:33,966 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5d5422cb62322ad966237/log.txt.gz 2026-03-27 00:55:34,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:55:34,737 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:55:34,739 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5d5332cb62322ad966224/log.txt.gz 2026-03-27 00:55:34,856 - lava_callback - INFO - Received callback for node 69c5d5362cb62322ad966227 INFO: 172.20.0.1:56760 - "POST /node/69c5d5362cb62322ad966227 HTTP/1.0" 202 Accepted 2026-03-27 00:55:34,932 - lava_callback - INFO - Starting async job submit for node 69c5d5362cb62322ad966227 2026-03-27 00:55:35,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:55:35,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5362cb62322ad966227 HTTP/1.1" 200 None 2026-03-27 00:55:35,100 - lava_callback - DEBUG - Processing node 69c5d5362cb62322ad966227: baseline-arm64 2026-03-27 00:55:35,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:55:35,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:55:35,881 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:55:35,884 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5d5422cb62322ad966237/lava-logs.yaml 2026-03-27 00:55:35,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:56774 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:55:36,872 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:55:36,875 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5d5332cb62322ad966224/lava-logs.yaml 2026-03-27 00:55:36,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:55:37,143 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:55:37,145 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5d5362cb62322ad966227/log.txt.gz 2026-03-27 00:55:37,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:55:37,736 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:55:37,738 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5d5422cb62322ad966237/lava_callback.json.gz 2026-03-27 00:55:37,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:55:37,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5422cb62322ad966237 HTTP/1.1" 200 None 2026-03-27 00:55:37,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:55:37,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-27 00:55:37,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:55:37,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d5422cb62322ad966237 HTTP/1.1" 200 None 2026-03-27 00:55:37,950 - lava_callback - INFO - Completed processing callback for node 69c5d5422cb62322ad966237 2026-03-27 00:55:38,809 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:55:38,812 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5d5332cb62322ad966224/lava_callback.json.gz 2026-03-27 00:55:38,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:55:38,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5332cb62322ad966224 HTTP/1.1" 200 None 2026-03-27 00:55:38,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:55:38,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-27 00:55:38,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:55:38,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d5332cb62322ad966224 HTTP/1.1" 200 None 2026-03-27 00:55:38,938 - lava_callback - INFO - Completed processing callback for node 69c5d5332cb62322ad966224 2026-03-27 00:55:39,347 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:55:39,349 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5d5362cb62322ad966227/lava-logs.yaml 2026-03-27 00:55:39,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:55:41,015 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:55:41,017 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5d5362cb62322ad966227/lava_callback.json.gz 2026-03-27 00:55:41,040 - lava_callback - INFO - Received callback for node 69c3328018a3717aa65d2b09 INFO: 172.20.0.1:40328 - "POST /node/69c3328018a3717aa65d2b09 HTTP/1.0" 202 Accepted 2026-03-27 00:55:41,058 - lava_callback - INFO - Starting async job submit for node 69c3328018a3717aa65d2b09 2026-03-27 00:55:41,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:55:41,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:55:41,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5362cb62322ad966227 HTTP/1.1" 200 None 2026-03-27 00:55:41,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:55:41,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3328018a3717aa65d2b09 HTTP/1.1" 200 None 2026-03-27 00:55:41,091 - lava_callback - DEBUG - Processing node 69c3328018a3717aa65d2b09: baseline-arm-clabbe 2026-03-27 00:55:41,091 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3328018a3717aa65d2b09 2026-03-27 00:55:41,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:55:41,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-27 00:55:41,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:55:41,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3328018a3717aa65d2b09 HTTP/1.1" 200 None 2026-03-27 00:55:41,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:55:41,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-27 00:55:41,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:55:41,158 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d5362cb62322ad966227 HTTP/1.1" 200 None 2026-03-27 00:55:41,168 - lava_callback - INFO - Completed processing callback for node 69c5d5362cb62322ad966227 2026-03-27 00:55:41,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3328018a3717aa65d2b09 HTTP/1.1" 200 None 2026-03-27 00:55:41,233 - lava_callback - ERROR - Error processing callback for node 69c3328018a3717aa65d2b09: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-27 00:55:45,014 - lava_callback - INFO - Received callback for node 69c5d53a2cb62322ad96622a INFO: 172.20.0.1:40334 - "POST /node/69c5d53a2cb62322ad96622a HTTP/1.0" 202 Accepted 2026-03-27 00:55:45,037 - lava_callback - INFO - Starting async job submit for node 69c5d53a2cb62322ad96622a 2026-03-27 00:55:45,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:55:45,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d53a2cb62322ad96622a HTTP/1.1" 200 None 2026-03-27 00:55:45,125 - lava_callback - DEBUG - Processing node 69c5d53a2cb62322ad96622a: baseline-arm64 2026-03-27 00:55:45,318 - lava_callback - WARNING - Failed to upload log 2026-03-27 00:55:45,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:40348 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:55:47,052 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:55:47,054 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5d53a2cb62322ad96622a/lava-logs.yaml 2026-03-27 00:55:47,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:55:47,757 - lava_callback - INFO - Received callback for node 69c5d5452cb62322ad966248 INFO: 172.20.0.1:40356 - "POST /node/69c5d5452cb62322ad966248 HTTP/1.0" 202 Accepted 2026-03-27 00:55:47,767 - lava_callback - INFO - Starting async job submit for node 69c5d5452cb62322ad966248 2026-03-27 00:55:47,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:55:47,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5452cb62322ad966248 HTTP/1.1" 200 None 2026-03-27 00:55:47,827 - lava_callback - DEBUG - Processing node 69c5d5452cb62322ad966248: baseline-arm64 2026-03-27 00:55:48,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:55:48,515 - lava_callback - INFO - Received callback for node 69c5d5762cb62322ad966298 INFO: 172.20.0.1:40366 - "POST /node/69c5d5762cb62322ad966298 HTTP/1.0" 202 Accepted 2026-03-27 00:55:48,525 - lava_callback - INFO - Starting async job submit for node 69c5d5762cb62322ad966298 2026-03-27 00:55:48,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:55:48,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5762cb62322ad966298 HTTP/1.1" 200 None 2026-03-27 00:55:48,547 - lava_callback - DEBUG - Processing node 69c5d5762cb62322ad966298: baseline-nfs-arm64 2026-03-27 00:55:48,549 - lava_callback - WARNING - Failed to upload log 2026-03-27 00:55:48,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:55:48,756 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:55:48,759 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5d53a2cb62322ad96622a/lava_callback.json.gz 2026-03-27 00:55:48,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:55:48,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d53a2cb62322ad96622a HTTP/1.1" 200 None 2026-03-27 00:55:48,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:55:48,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-27 00:55:48,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:55:48,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d53a2cb62322ad96622a HTTP/1.1" 200 None 2026-03-27 00:55:48,986 - lava_callback - INFO - Completed processing callback for node 69c5d53a2cb62322ad96622a 2026-03-27 00:55:50,076 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:55:50,078 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5d5452cb62322ad966248/log.txt.gz 2026-03-27 00:55:50,245 - lava_callback - INFO - Received callback for node 69c5d5312cb62322ad966221 INFO: 172.20.0.1:33506 - "POST /node/69c5d5312cb62322ad966221 HTTP/1.0" 202 Accepted 2026-03-27 00:55:50,276 - lava_callback - INFO - Starting async job submit for node 69c5d5312cb62322ad966221 2026-03-27 00:55:50,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:55:50,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5312cb62322ad966221 HTTP/1.1" 200 None 2026-03-27 00:55:50,461 - lava_callback - DEBUG - Processing node 69c5d5312cb62322ad966221: baseline-arm64 2026-03-27 00:55:50,461 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:55:50,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:55:50,541 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5d5762cb62322ad966298/lava-logs.yaml 2026-03-27 00:55:50,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:55:50,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:55:50,879 - lava_callback - INFO - Received callback for node 69c5d53d2cb62322ad96622e INFO: 172.20.0.1:33516 - "POST /node/69c5d53d2cb62322ad96622e HTTP/1.0" 202 Accepted 2026-03-27 00:55:50,889 - lava_callback - INFO - Starting async job submit for node 69c5d53d2cb62322ad96622e 2026-03-27 00:55:50,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:55:50,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d53d2cb62322ad96622e HTTP/1.1" 200 None 2026-03-27 00:55:50,947 - lava_callback - DEBUG - Processing node 69c5d53d2cb62322ad96622e: baseline-arm64 2026-03-27 00:55:51,164 - lava_callback - WARNING - Failed to upload log 2026-03-27 00:55:51,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:55:51,636 - lava_callback - INFO - Received callback for node 69c5d55e2cb62322ad966268 2026-03-27 00:55:51,646 - lava_callback - INFO - Starting async job submit for node 69c5d55e2cb62322ad966268 INFO: 172.20.0.1:33520 - "POST /node/69c5d55e2cb62322ad966268 HTTP/1.0" 202 Accepted 2026-03-27 00:55:51,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:55:51,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d55e2cb62322ad966268 HTTP/1.1" 200 None 2026-03-27 00:55:51,703 - lava_callback - DEBUG - Processing node 69c5d55e2cb62322ad966268: baseline-arm64 2026-03-27 00:55:52,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:55:52,171 - lava_callback - INFO - Received callback for node 69c5d55b2cb62322ad966265 2026-03-27 00:55:52,181 - lava_callback - INFO - Starting async job submit for node 69c5d55b2cb62322ad966265 INFO: 172.20.0.1:33528 - "POST /node/69c5d55b2cb62322ad966265 HTTP/1.0" 202 Accepted 2026-03-27 00:55:52,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:55:52,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d55b2cb62322ad966265 HTTP/1.1" 200 None 2026-03-27 00:55:52,240 - lava_callback - DEBUG - Processing node 69c5d55b2cb62322ad966265: baseline-arm64 2026-03-27 00:55:52,358 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:55:52,389 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:55:52,421 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5d5762cb62322ad966298/lava_callback.json.gz 2026-03-27 00:55:52,424 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5d5452cb62322ad966248/lava-logs.yaml 2026-03-27 00:55:52,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:55:52,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:55:52,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5762cb62322ad966298 HTTP/1.1" 200 None 2026-03-27 00:55:52,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:55:52,624 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:55:52,637 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5d5312cb62322ad966221/log.txt.gz 2026-03-27 00:55:52,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-27 00:55:52,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:55:52,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:55:52,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d5762cb62322ad966298 HTTP/1.1" 200 None 2026-03-27 00:55:52,847 - lava_callback - INFO - Completed processing callback for node 69c5d5762cb62322ad966298 2026-03-27 00:55:53,076 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:55:53,338 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:55:53,340 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5d53d2cb62322ad96622e/lava-logs.yaml 2026-03-27 00:55:53,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:55:54,152 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:55:54,154 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5d55e2cb62322ad966268/log.txt.gz 2026-03-27 00:55:54,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:55:54,631 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:55:54,633 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5d5452cb62322ad966248/lava_callback.json.gz 2026-03-27 00:55:54,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:55:54,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5452cb62322ad966248 HTTP/1.1" 200 None 2026-03-27 00:55:54,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:55:54,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-27 00:55:54,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:55:54,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d5452cb62322ad966248 HTTP/1.1" 200 None 2026-03-27 00:55:54,780 - lava_callback - INFO - Completed processing callback for node 69c5d5452cb62322ad966248 2026-03-27 00:55:54,896 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:55:54,898 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5d55b2cb62322ad966265/log.txt.gz 2026-03-27 00:55:55,066 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:55:55,095 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5d5312cb62322ad966221/lava-logs.yaml 2026-03-27 00:55:55,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:55:55,147 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:55:55,242 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:55:55,244 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5d53d2cb62322ad96622e/lava_callback.json.gz 2026-03-27 00:55:55,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:55:55,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d53d2cb62322ad96622e HTTP/1.1" 200 None 2026-03-27 00:55:55,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:55:55,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-27 00:55:55,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:55:55,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d53d2cb62322ad96622e HTTP/1.1" 200 None 2026-03-27 00:55:55,424 - lava_callback - INFO - Completed processing callback for node 69c5d53d2cb62322ad96622e 2026-03-27 00:55:56,186 - lava_callback - INFO - Received callback for node 69c5d54d2cb62322ad96624e INFO: 172.20.0.1:33536 - "POST /node/69c5d54d2cb62322ad96624e HTTP/1.0" 202 Accepted 2026-03-27 00:55:56,192 - lava_callback - INFO - Starting async job submit for node 69c5d54d2cb62322ad96624e 2026-03-27 00:55:56,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:55:56,247 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d54d2cb62322ad96624e HTTP/1.1" 200 None 2026-03-27 00:55:56,248 - lava_callback - DEBUG - Processing node 69c5d54d2cb62322ad96624e: baseline-arm64 INFO: 172.20.0.1:33546 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:55:56,680 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:55:56,688 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5d55e2cb62322ad966268/lava-logs.yaml 2026-03-27 00:55:56,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:55:56,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:55:57,063 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:55:57,068 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5d5312cb62322ad966221/lava_callback.json.gz 2026-03-27 00:55:57,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:55:57,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5312cb62322ad966221 HTTP/1.1" 200 None 2026-03-27 00:55:57,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:55:57,164 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-27 00:55:57,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:55:57,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d5312cb62322ad966221 HTTP/1.1" 200 None 2026-03-27 00:55:57,210 - lava_callback - INFO - Completed processing callback for node 69c5d5312cb62322ad966221 2026-03-27 00:55:57,299 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:55:57,302 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5d55b2cb62322ad966265/lava-logs.yaml 2026-03-27 00:55:57,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:55:58,685 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:55:58,687 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5d55e2cb62322ad966268/lava_callback.json.gz 2026-03-27 00:55:58,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:55:58,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d55e2cb62322ad966268 HTTP/1.1" 200 None 2026-03-27 00:55:58,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:55:58,750 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:55:58,753 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5d54d2cb62322ad96624e/log.txt.gz 2026-03-27 00:55:58,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-27 00:55:58,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:55:58,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d55e2cb62322ad966268 HTTP/1.1" 200 None 2026-03-27 00:55:58,976 - lava_callback - INFO - Completed processing callback for node 69c5d55e2cb62322ad966268 2026-03-27 00:55:59,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:55:59,277 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:55:59,280 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5d55b2cb62322ad966265/lava_callback.json.gz 2026-03-27 00:55:59,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:55:59,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d55b2cb62322ad966265 HTTP/1.1" 200 None 2026-03-27 00:55:59,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:55:59,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-27 00:55:59,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:55:59,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d55b2cb62322ad966265 HTTP/1.1" 200 None 2026-03-27 00:55:59,377 - lava_callback - INFO - Completed processing callback for node 69c5d55b2cb62322ad966265 2026-03-27 00:55:59,852 - lava_callback - INFO - Received callback for node 69c5d5482cb62322ad96624b INFO: 172.20.0.1:59028 - "POST /node/69c5d5482cb62322ad96624b HTTP/1.0" 202 Accepted 2026-03-27 00:55:59,862 - lava_callback - INFO - Starting async job submit for node 69c5d5482cb62322ad96624b 2026-03-27 00:55:59,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:55:59,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:55:59,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5482cb62322ad96624b HTTP/1.1" 200 None 2026-03-27 00:55:59,934 - lava_callback - DEBUG - Processing node 69c5d5482cb62322ad96624b: baseline-arm64 2026-03-27 00:55:59,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-27 00:56:00,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:56:01,061 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:56:01,064 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5d54d2cb62322ad96624e/lava-logs.yaml 2026-03-27 00:56:01,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:56:01,874 - lava_callback - INFO - Received callback for node 69c5d5582cb62322ad966262 INFO: 172.20.0.1:59044 - "POST /node/69c5d5582cb62322ad966262 HTTP/1.0" 202 Accepted 2026-03-27 00:56:01,885 - lava_callback - INFO - Starting async job submit for node 69c5d5582cb62322ad966262 2026-03-27 00:56:01,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:56:01,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5582cb62322ad966262 HTTP/1.1" 200 None 2026-03-27 00:56:01,940 - lava_callback - DEBUG - Processing node 69c5d5582cb62322ad966262: baseline-arm64 2026-03-27 00:56:02,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:56:02,482 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:56:02,484 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5d5482cb62322ad96624b/log.txt.gz 2026-03-27 00:56:02,503 - lava_callback - INFO - Received callback for node 69c5d5512cb62322ad966258 INFO: 172.20.0.1:59050 - "POST /node/69c5d5512cb62322ad966258 HTTP/1.0" 202 Accepted 2026-03-27 00:56:02,531 - lava_callback - INFO - Starting async job submit for node 69c5d5512cb62322ad966258 2026-03-27 00:56:02,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:56:02,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5512cb62322ad966258 HTTP/1.1" 200 None 2026-03-27 00:56:02,703 - lava_callback - DEBUG - Processing node 69c5d5512cb62322ad966258: baseline-arm64 2026-03-27 00:56:03,030 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:56:03,118 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:56:03,176 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5d54d2cb62322ad96624e/lava_callback.json.gz 2026-03-27 00:56:03,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:56:03,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:56:03,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d54d2cb62322ad96624e HTTP/1.1" 200 None 2026-03-27 00:56:03,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:56:03,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-27 00:56:03,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:56:03,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d54d2cb62322ad96624e HTTP/1.1" 200 None 2026-03-27 00:56:03,427 - lava_callback - INFO - Completed processing callback for node 69c5d54d2cb62322ad96624e 2026-03-27 00:56:04,522 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:56:04,524 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5d5582cb62322ad966262/log.txt.gz 2026-03-27 00:56:04,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:56:05,011 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:56:05,013 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5d5482cb62322ad96624b/lava-logs.yaml 2026-03-27 00:56:05,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:56:05,099 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:56:05,101 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5d5512cb62322ad966258/log.txt.gz 2026-03-27 00:56:05,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:59066 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:56:06,561 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:56:06,563 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5d5582cb62322ad966262/lava-logs.yaml 2026-03-27 00:56:06,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:56:06,800 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:56:06,802 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5d5482cb62322ad96624b/lava_callback.json.gz 2026-03-27 00:56:06,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:56:06,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5482cb62322ad96624b HTTP/1.1" 200 None 2026-03-27 00:56:06,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:56:06,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-27 00:56:06,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:56:06,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d5482cb62322ad96624b HTTP/1.1" 200 None 2026-03-27 00:56:06,901 - lava_callback - INFO - Completed processing callback for node 69c5d5482cb62322ad96624b 2026-03-27 00:56:07,105 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:56:07,107 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5d5512cb62322ad966258/lava-logs.yaml 2026-03-27 00:56:07,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:56:08,379 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:56:08,381 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5d5582cb62322ad966262/lava_callback.json.gz 2026-03-27 00:56:08,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:56:08,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5582cb62322ad966262 HTTP/1.1" 200 None 2026-03-27 00:56:08,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:56:08,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-27 00:56:08,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:56:08,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d5582cb62322ad966262 HTTP/1.1" 200 None 2026-03-27 00:56:08,517 - lava_callback - INFO - Completed processing callback for node 69c5d5582cb62322ad966262 2026-03-27 00:56:08,924 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:56:08,926 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5d5512cb62322ad966258/lava_callback.json.gz 2026-03-27 00:56:08,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:56:08,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5512cb62322ad966258 HTTP/1.1" 200 None 2026-03-27 00:56:08,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:56:08,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-27 00:56:08,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:56:09,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d5512cb62322ad966258 HTTP/1.1" 200 None 2026-03-27 00:56:09,037 - lava_callback - INFO - Completed processing callback for node 69c5d5512cb62322ad966258 INFO: 172.20.0.1:54490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54506 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54510 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:56:21,046 - lava_callback - INFO - Received callback for node 69c332a618a3717aa65d2b31 INFO: 172.20.0.1:59610 - "POST /node/69c332a618a3717aa65d2b31 HTTP/1.0" 202 Accepted 2026-03-27 00:56:21,060 - lava_callback - INFO - Starting async job submit for node 69c332a618a3717aa65d2b31 2026-03-27 00:56:21,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:56:21,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c332a618a3717aa65d2b31 HTTP/1.1" 200 None 2026-03-27 00:56:21,090 - lava_callback - DEBUG - Processing node 69c332a618a3717aa65d2b31: baseline-arm-clabbe 2026-03-27 00:56:21,090 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c332a618a3717aa65d2b31 2026-03-27 00:56:21,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:56:21,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c332a618a3717aa65d2b31 HTTP/1.1" 200 None 2026-03-27 00:56:21,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:56:21,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-27 00:56:21,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:56:21,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c332a618a3717aa65d2b31 HTTP/1.1" 200 None 2026-03-27 00:56:21,169 - lava_callback - ERROR - Error processing callback for node 69c332a618a3717aa65d2b31: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59612 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:56:29,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:56:29,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-27 00:56:31,830 - lava_callback - INFO - Received callback for node 69c5d5702cb62322ad966291 INFO: 172.20.0.1:46544 - "POST /node/69c5d5702cb62322ad966291 HTTP/1.0" 202 Accepted 2026-03-27 00:56:31,840 - lava_callback - INFO - Starting async job submit for node 69c5d5702cb62322ad966291 2026-03-27 00:56:31,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:56:31,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5702cb62322ad966291 HTTP/1.1" 200 None 2026-03-27 00:56:31,910 - lava_callback - DEBUG - Processing node 69c5d5702cb62322ad966291: baseline-nfs-arm64 2026-03-27 00:56:32,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:56:36,185 - lava_callback - INFO - Received callback for node 69c5d5732cb62322ad966295 INFO: 172.20.0.1:46558 - "POST /node/69c5d5732cb62322ad966295 HTTP/1.0" 202 Accepted 2026-03-27 00:56:36,196 - lava_callback - INFO - Starting async job submit for node 69c5d5732cb62322ad966295 2026-03-27 00:56:36,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:56:36,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5732cb62322ad966295 HTTP/1.1" 200 None 2026-03-27 00:56:36,243 - lava_callback - DEBUG - Processing node 69c5d5732cb62322ad966295: baseline-nfs-arm64 INFO: 172.20.0.1:46562 - "GET / HTTP/1.0" 200 OK Upload attempt 1 failed with ConnectionError: HTTPSConnectionPool(host='files-staging.kernelci.org', port=443): Max retries exceeded with url: /upload (Caused by NameResolutionError(": Failed to resolve 'files-staging.kernelci.org' ([Errno -2] Name or service not known)")) Retrying in 10 seconds... (4 retries remaining) 2026-03-27 00:56:37,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:56:38,170 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:56:38,172 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5d5732cb62322ad966295/log.txt.gz 2026-03-27 00:56:38,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:56:39,684 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:56:39,687 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5d5732cb62322ad966295/lava-logs.yaml 2026-03-27 00:56:39,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:56:40,777 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:56:40,780 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5d5732cb62322ad966295/lava_callback.json.gz 2026-03-27 00:56:40,819 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:56:40,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5732cb62322ad966295 HTTP/1.1" 200 None 2026-03-27 00:56:40,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:56:40,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-27 00:56:40,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:56:40,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d5732cb62322ad966295 HTTP/1.1" 200 None 2026-03-27 00:56:40,917 - lava_callback - INFO - Completed processing callback for node 69c5d5732cb62322ad966295 2026-03-27 00:56:42,289 - lava_callback - INFO - Received callback for node 69c5d57f2cb62322ad9662a5 INFO: 172.20.0.1:44454 - "POST /node/69c5d57f2cb62322ad9662a5 HTTP/1.0" 202 Accepted 2026-03-27 00:56:42,304 - lava_callback - INFO - Starting async job submit for node 69c5d57f2cb62322ad9662a5 2026-03-27 00:56:42,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:56:42,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d57f2cb62322ad9662a5 HTTP/1.1" 200 None 2026-03-27 00:56:42,390 - lava_callback - DEBUG - Processing node 69c5d57f2cb62322ad9662a5: baseline-nfs-arm64 2026-03-27 00:56:42,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:56:43,946 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:56:43,948 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5d57f2cb62322ad9662a5/log.txt.gz 2026-03-27 00:56:44,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:56:45,213 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:56:45,216 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5d57f2cb62322ad9662a5/lava-logs.yaml 2026-03-27 00:56:45,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:56:46,303 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:56:46,305 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5d57f2cb62322ad9662a5/lava_callback.json.gz 2026-03-27 00:56:46,339 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:56:46,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d57f2cb62322ad9662a5 HTTP/1.1" 200 None 2026-03-27 00:56:46,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:56:46,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-27 00:56:46,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:56:46,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d57f2cb62322ad9662a5 HTTP/1.1" 200 None 2026-03-27 00:56:46,491 - lava_callback - INFO - Completed processing callback for node 69c5d57f2cb62322ad9662a5 INFO: 172.20.0.1:44460 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:56:46,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:56:48,087 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:56:48,088 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5d5702cb62322ad966291/log.txt.gz 2026-03-27 00:56:48,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:56:49,683 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:56:49,686 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5d5702cb62322ad966291/lava-logs.yaml 2026-03-27 00:56:49,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:56:49,862 - lava_callback - INFO - Received callback for node 69c5d56a2cb62322ad966287 INFO: 172.20.0.1:51038 - "POST /node/69c5d56a2cb62322ad966287 HTTP/1.0" 202 Accepted 2026-03-27 00:56:49,873 - lava_callback - INFO - Starting async job submit for node 69c5d56a2cb62322ad966287 2026-03-27 00:56:49,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:56:49,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d56a2cb62322ad966287 HTTP/1.1" 200 None 2026-03-27 00:56:49,919 - lava_callback - DEBUG - Processing node 69c5d56a2cb62322ad966287: baseline-nfs-arm64 2026-03-27 00:56:50,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:56:50,946 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:56:50,948 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5d5702cb62322ad966291/lava_callback.json.gz 2026-03-27 00:56:51,030 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:56:51,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5702cb62322ad966291 HTTP/1.1" 200 None 2026-03-27 00:56:51,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:56:51,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-27 00:56:51,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:56:51,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d5702cb62322ad966291 HTTP/1.1" 200 None 2026-03-27 00:56:51,130 - lava_callback - INFO - Completed processing callback for node 69c5d5702cb62322ad966291 2026-03-27 00:56:51,861 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:56:51,862 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5d56a2cb62322ad966287/log.txt.gz 2026-03-27 00:56:52,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:56:53,376 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:56:53,379 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5d56a2cb62322ad966287/lava-logs.yaml 2026-03-27 00:56:53,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:56:54,523 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:56:54,525 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5d56a2cb62322ad966287/lava_callback.json.gz 2026-03-27 00:56:54,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:56:54,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d56a2cb62322ad966287 HTTP/1.1" 200 None 2026-03-27 00:56:54,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:56:54,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-27 00:56:54,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:56:54,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d56a2cb62322ad966287 HTTP/1.1" 200 None 2026-03-27 00:56:54,645 - lava_callback - INFO - Completed processing callback for node 69c5d56a2cb62322ad966287 INFO: 172.20.0.1:51054 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:56:59,676 - lava_callback - INFO - Received callback for node 69c5d5652cb62322ad966280 INFO: 172.20.0.1:51070 - "POST /node/69c5d5652cb62322ad966280 HTTP/1.0" 202 Accepted 2026-03-27 00:56:59,687 - lava_callback - INFO - Starting async job submit for node 69c5d5652cb62322ad966280 2026-03-27 00:56:59,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:56:59,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5652cb62322ad966280 HTTP/1.1" 200 None 2026-03-27 00:56:59,756 - lava_callback - DEBUG - Processing node 69c5d5652cb62322ad966280: baseline-nfs-arm64 2026-03-27 00:56:59,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:57:00,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-27 00:57:00,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:57:00,839 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:57:00,841 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5d5652cb62322ad966280/log.txt.gz 2026-03-27 00:57:01,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:57:01,715 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:57:01,717 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5d5652cb62322ad966280/lava-logs.yaml 2026-03-27 00:57:01,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:57:01,801 - lava_callback - INFO - Received callback for node 69c5d57c2cb62322ad9662a2 INFO: 172.20.0.1:52658 - "POST /node/69c5d57c2cb62322ad9662a2 HTTP/1.0" 202 Accepted 2026-03-27 00:57:01,809 - lava_callback - INFO - Starting async job submit for node 69c5d57c2cb62322ad9662a2 2026-03-27 00:57:01,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:57:01,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d57c2cb62322ad9662a2 HTTP/1.1" 200 None 2026-03-27 00:57:01,853 - lava_callback - DEBUG - Processing node 69c5d57c2cb62322ad9662a2: baseline-nfs-arm64 2026-03-27 00:57:02,287 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:57:02,324 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5d5652cb62322ad966280/lava_callback.json.gz 2026-03-27 00:57:02,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:57:02,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:57:02,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5652cb62322ad966280 HTTP/1.1" 200 None 2026-03-27 00:57:02,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:57:02,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-27 00:57:02,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:57:02,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d5652cb62322ad966280 HTTP/1.1" 200 None 2026-03-27 00:57:02,480 - lava_callback - INFO - Completed processing callback for node 69c5d5652cb62322ad966280 2026-03-27 00:57:02,935 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:57:02,937 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5d57c2cb62322ad9662a2/log.txt.gz 2026-03-27 00:57:03,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:57:03,913 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:57:03,916 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5d57c2cb62322ad9662a2/lava-logs.yaml 2026-03-27 00:57:03,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:57:04,615 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:57:04,617 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5d57c2cb62322ad9662a2/lava_callback.json.gz 2026-03-27 00:57:04,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:57:04,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d57c2cb62322ad9662a2 HTTP/1.1" 200 None 2026-03-27 00:57:04,665 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:57:04,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-27 00:57:04,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:57:04,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d57c2cb62322ad9662a2 HTTP/1.1" 200 None 2026-03-27 00:57:04,727 - lava_callback - INFO - Completed processing callback for node 69c5d57c2cb62322ad9662a2 INFO: 172.20.0.1:52664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40370 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-27 00:57:15,532 - lava_callback - INFO - Received callback for node 69c5d5772cb62322ad96629b INFO: 172.20.0.1:40382 - "POST /node/69c5d5772cb62322ad96629b HTTP/1.0" 202 Accepted 2026-03-27 00:57:15,543 - lava_callback - INFO - Starting async job submit for node 69c5d5772cb62322ad96629b 2026-03-27 00:57:15,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:57:15,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5772cb62322ad96629b HTTP/1.1" 200 None 2026-03-27 00:57:15,605 - lava_callback - DEBUG - Processing node 69c5d5772cb62322ad96629b: baseline-nfs-arm64 2026-03-27 00:57:16,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:40396 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:57:16,567 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:57:16,569 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5d5772cb62322ad96629b/log.txt.gz 2026-03-27 00:57:16,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:57:17,448 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:57:17,451 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5d5772cb62322ad96629b/lava-logs.yaml 2026-03-27 00:57:17,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:57:17,649 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:57:17,651 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5d5772cb62322ad96629b/lava_callback.json.gz 2026-03-27 00:57:17,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:57:17,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5772cb62322ad96629b HTTP/1.1" 200 None 2026-03-27 00:57:17,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:57:17,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-27 00:57:17,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:57:17,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d5772cb62322ad96629b HTTP/1.1" 200 None 2026-03-27 00:57:17,744 - lava_callback - INFO - Completed processing callback for node 69c5d5772cb62322ad96629b 2026-03-27 00:57:19,113 - lava_callback - INFO - Received callback for node 69c5d5822cb62322ad9662c1 INFO: 172.20.0.1:40408 - "POST /node/69c5d5822cb62322ad9662c1 HTTP/1.0" 202 Accepted 2026-03-27 00:57:19,120 - lava_callback - INFO - Starting async job submit for node 69c5d5822cb62322ad9662c1 2026-03-27 00:57:19,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:57:19,158 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5822cb62322ad9662c1 HTTP/1.1" 200 None 2026-03-27 00:57:19,158 - lava_callback - DEBUG - Processing node 69c5d5822cb62322ad9662c1: baseline-nfs-arm64 2026-03-27 00:57:19,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:57:19,703 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:57:19,704 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5d5822cb62322ad9662c1/log.txt.gz 2026-03-27 00:57:19,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:57:20,138 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:57:20,141 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5d5822cb62322ad9662c1/lava-logs.yaml 2026-03-27 00:57:20,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:57:20,302 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:57:20,304 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5d5822cb62322ad9662c1/lava_callback.json.gz 2026-03-27 00:57:20,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:57:20,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5822cb62322ad9662c1 HTTP/1.1" 200 None 2026-03-27 00:57:20,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:57:20,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-27 00:57:20,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:57:20,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d5822cb62322ad9662c1 HTTP/1.1" 200 None 2026-03-27 00:57:20,397 - lava_callback - INFO - Completed processing callback for node 69c5d5822cb62322ad9662c1 INFO: 172.20.0.1:46098 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:57:30,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:57:30,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.20.0.1:38896 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:57:43,590 - lava_callback - INFO - Received callback for node 69c5d5c92cb62322ad966386 INFO: 172.20.0.1:51702 - "POST /node/69c5d5c92cb62322ad966386 HTTP/1.0" 202 Accepted 2026-03-27 00:57:43,596 - lava_callback - INFO - Starting async job submit for node 69c5d5c92cb62322ad966386 2026-03-27 00:57:43,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:57:43,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5c92cb62322ad966386 HTTP/1.1" 200 None 2026-03-27 00:57:43,640 - lava_callback - DEBUG - Processing node 69c5d5c92cb62322ad966386: baseline-arm64-pengutronix 2026-03-27 00:57:43,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:57:44,010 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:57:44,012 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c5d5c92cb62322ad966386/log.txt.gz 2026-03-27 00:57:44,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:57:44,345 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:57:44,347 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c5d5c92cb62322ad966386/lava-logs.yaml 2026-03-27 00:57:44,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:57:44,653 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:57:44,655 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c5d5c92cb62322ad966386/lava_callback.json.gz 2026-03-27 00:57:44,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:57:44,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5c92cb62322ad966386 HTTP/1.1" 200 None 2026-03-27 00:57:44,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:57:44,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-27 00:57:44,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:57:44,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d5c92cb62322ad966386 HTTP/1.1" 200 None 2026-03-27 00:57:44,752 - lava_callback - INFO - Completed processing callback for node 69c5d5c92cb62322ad966386 INFO: 172.20.0.1:51710 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:57:47,140 - lava_callback - INFO - Received callback for node 69c5d5d12cb62322ad966393 INFO: 172.20.0.1:51722 - "POST /node/69c5d5d12cb62322ad966393 HTTP/1.0" 202 Accepted 2026-03-27 00:57:47,146 - lava_callback - INFO - Starting async job submit for node 69c5d5d12cb62322ad966393 2026-03-27 00:57:47,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:57:47,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5d12cb62322ad966393 HTTP/1.1" 200 None 2026-03-27 00:57:47,187 - lava_callback - DEBUG - Processing node 69c5d5d12cb62322ad966393: baseline-arm64-foundriesio 2026-03-27 00:57:47,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:57:47,509 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:57:47,511 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c5d5d12cb62322ad966393/log.txt.gz 2026-03-27 00:57:47,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:57:47,813 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:57:47,815 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c5d5d12cb62322ad966393/lava-logs.yaml 2026-03-27 00:57:47,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:57:47,954 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:57:47,955 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c5d5d12cb62322ad966393/lava_callback.json.gz 2026-03-27 00:57:47,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:57:47,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5d12cb62322ad966393 HTTP/1.1" 200 None 2026-03-27 00:57:47,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:57:48,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-27 00:57:48,015 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:57:48,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d5d12cb62322ad966393 HTTP/1.1" 200 None 2026-03-27 00:57:48,044 - lava_callback - INFO - Completed processing callback for node 69c5d5d12cb62322ad966393 2026-03-27 00:57:52,103 - lava_callback - INFO - Received callback for node 69c5d5cc2cb62322ad966389 INFO: 172.20.0.1:60232 - "POST /node/69c5d5cc2cb62322ad966389 HTTP/1.0" 202 Accepted 2026-03-27 00:57:52,110 - lava_callback - INFO - Starting async job submit for node 69c5d5cc2cb62322ad966389 2026-03-27 00:57:52,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:57:52,149 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5cc2cb62322ad966389 HTTP/1.1" 200 None 2026-03-27 00:57:52,150 - lava_callback - DEBUG - Processing node 69c5d5cc2cb62322ad966389: baseline-arm64-pengutronix 2026-03-27 00:57:52,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:57:52,611 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:57:52,613 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c5d5cc2cb62322ad966389/log.txt.gz 2026-03-27 00:57:52,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:57:52,993 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:57:52,994 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c5d5cc2cb62322ad966389/lava-logs.yaml 2026-03-27 00:57:53,000 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:57:53,135 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:57:53,137 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c5d5cc2cb62322ad966389/lava_callback.json.gz 2026-03-27 00:57:53,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:57:53,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5cc2cb62322ad966389 HTTP/1.1" 200 None 2026-03-27 00:57:53,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:57:53,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-27 00:57:53,187 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:57:53,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d5cc2cb62322ad966389 HTTP/1.1" 200 None 2026-03-27 00:57:53,217 - lava_callback - INFO - Completed processing callback for node 69c5d5cc2cb62322ad966389 INFO: 172.20.0.1:60238 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:58:00,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:58:00,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-27 00:58:05,471 - lava_callback - INFO - Received callback for node 69c5d5b42cb62322ad966347 INFO: 172.20.0.1:37744 - "POST /node/69c5d5b42cb62322ad966347 HTTP/1.0" 202 Accepted 2026-03-27 00:58:05,477 - lava_callback - INFO - Starting async job submit for node 69c5d5b42cb62322ad966347 2026-03-27 00:58:05,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:58:05,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5b42cb62322ad966347 HTTP/1.1" 200 None 2026-03-27 00:58:05,527 - lava_callback - DEBUG - Processing node 69c5d5b42cb62322ad966347: baseline-arm64-broonie 2026-03-27 00:58:05,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:58:06,086 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:58:06,088 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c5d5b42cb62322ad966347/log.txt.gz 2026-03-27 00:58:06,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:58:06,436 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:58:06,438 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c5d5b42cb62322ad966347/lava-logs.yaml 2026-03-27 00:58:06,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:37756 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:58:06,571 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:58:06,573 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c5d5b42cb62322ad966347/lava_callback.json.gz 2026-03-27 00:58:06,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:58:06,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5b42cb62322ad966347 HTTP/1.1" 200 None 2026-03-27 00:58:06,611 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:58:06,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-27 00:58:06,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:58:06,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d5b42cb62322ad966347 HTTP/1.1" 200 None 2026-03-27 00:58:06,651 - lava_callback - INFO - Completed processing callback for node 69c5d5b42cb62322ad966347 INFO: 172.20.0.1:47914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47922 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47932 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:58:18,847 - lava_callback - INFO - Received callback for node 69c5d5ce2cb62322ad966390 INFO: 172.20.0.1:47938 - "POST /node/69c5d5ce2cb62322ad966390 HTTP/1.0" 202 Accepted 2026-03-27 00:58:18,853 - lava_callback - INFO - Starting async job submit for node 69c5d5ce2cb62322ad966390 2026-03-27 00:58:18,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:58:18,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5ce2cb62322ad966390 HTTP/1.1" 200 None 2026-03-27 00:58:18,895 - lava_callback - DEBUG - Processing node 69c5d5ce2cb62322ad966390: baseline-arm64-pengutronix 2026-03-27 00:58:19,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:58:19,228 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:58:19,230 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c5d5ce2cb62322ad966390/log.txt.gz 2026-03-27 00:58:19,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:58:19,538 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:58:19,540 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c5d5ce2cb62322ad966390/lava-logs.yaml 2026-03-27 00:58:19,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:58:19,690 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:58:19,692 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c5d5ce2cb62322ad966390/lava_callback.json.gz 2026-03-27 00:58:19,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:58:19,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5ce2cb62322ad966390 HTTP/1.1" 200 None 2026-03-27 00:58:19,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:58:19,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-27 00:58:19,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:58:19,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d5ce2cb62322ad966390 HTTP/1.1" 200 None 2026-03-27 00:58:19,774 - lava_callback - INFO - Completed processing callback for node 69c5d5ce2cb62322ad966390 2026-03-27 00:58:24,447 - lava_callback - INFO - Received callback for node 69c5d5a72cb62322ad966333 INFO: 172.20.0.1:35664 - "POST /node/69c5d5a72cb62322ad966333 HTTP/1.0" 202 Accepted 2026-03-27 00:58:24,453 - lava_callback - INFO - Starting async job submit for node 69c5d5a72cb62322ad966333 2026-03-27 00:58:24,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:58:24,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5a72cb62322ad966333 HTTP/1.1" 200 None 2026-03-27 00:58:24,493 - lava_callback - DEBUG - Processing node 69c5d5a72cb62322ad966333: baseline-arm64-broonie 2026-03-27 00:58:24,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:58:24,915 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:58:24,917 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c5d5a72cb62322ad966333/log.txt.gz 2026-03-27 00:58:25,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:58:25,274 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:58:25,276 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c5d5a72cb62322ad966333/lava-logs.yaml 2026-03-27 00:58:25,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:58:25,451 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:58:25,453 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c5d5a72cb62322ad966333/lava_callback.json.gz 2026-03-27 00:58:25,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:58:25,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5a72cb62322ad966333 HTTP/1.1" 200 None 2026-03-27 00:58:25,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:58:25,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-27 00:58:25,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:58:25,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d5a72cb62322ad966333 HTTP/1.1" 200 None 2026-03-27 00:58:25,534 - lava_callback - INFO - Completed processing callback for node 69c5d5a72cb62322ad966333 2026-03-27 00:58:25,970 - lava_callback - INFO - Received callback for node 69c5d5b72cb62322ad96634e INFO: 172.20.0.1:35668 - "POST /node/69c5d5b72cb62322ad96634e HTTP/1.0" 202 Accepted 2026-03-27 00:58:25,976 - lava_callback - INFO - Starting async job submit for node 69c5d5b72cb62322ad96634e 2026-03-27 00:58:26,001 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:58:26,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5b72cb62322ad96634e HTTP/1.1" 200 None 2026-03-27 00:58:26,016 - lava_callback - DEBUG - Processing node 69c5d5b72cb62322ad96634e: baseline-arm64-broonie 2026-03-27 00:58:26,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:58:26,398 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:58:26,400 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c5d5b72cb62322ad96634e/log.txt.gz INFO: 172.20.0.1:35684 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:58:26,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:58:26,744 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:58:26,746 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c5d5b72cb62322ad96634e/lava-logs.yaml 2026-03-27 00:58:26,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:58:26,872 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:58:26,874 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c5d5b72cb62322ad96634e/lava_callback.json.gz 2026-03-27 00:58:26,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:58:26,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5b72cb62322ad96634e HTTP/1.1" 200 None 2026-03-27 00:58:26,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:58:26,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-27 00:58:26,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:58:26,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d5b72cb62322ad96634e HTTP/1.1" 200 None 2026-03-27 00:58:26,957 - lava_callback - INFO - Completed processing callback for node 69c5d5b72cb62322ad96634e 2026-03-27 00:58:30,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:58:30,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.20.0.1:48230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34816 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34492 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:59:28,338 - lava_callback - INFO - Received callback for node 69c3335c18a3717aa65d2b82 INFO: 172.20.0.1:34494 - "POST /node/69c3335c18a3717aa65d2b82 HTTP/1.0" 202 Accepted 2026-03-27 00:59:28,344 - lava_callback - INFO - Starting async job submit for node 69c3335c18a3717aa65d2b82 2026-03-27 00:59:28,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:59:28,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3335c18a3717aa65d2b82 HTTP/1.1" 200 None 2026-03-27 00:59:28,360 - lava_callback - DEBUG - Processing node 69c3335c18a3717aa65d2b82: baseline-arm-baylibre 2026-03-27 00:59:28,360 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3335c18a3717aa65d2b82 2026-03-27 00:59:28,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:59:28,374 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3335c18a3717aa65d2b82 HTTP/1.1" 200 None 2026-03-27 00:59:28,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:59:28,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-27 00:59:28,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:59:28,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3335c18a3717aa65d2b82 HTTP/1.1" 200 None 2026-03-27 00:59:28,419 - lava_callback - ERROR - Error processing callback for node 69c3335c18a3717aa65d2b82: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55268 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:59:48,263 - lava_callback - INFO - Received callback for node 69c3338218a3717aa65d2b8d INFO: 172.20.0.1:55276 - "POST /node/69c3338218a3717aa65d2b8d HTTP/1.0" 202 Accepted 2026-03-27 00:59:48,269 - lava_callback - INFO - Starting async job submit for node 69c3338218a3717aa65d2b8d 2026-03-27 00:59:48,270 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:59:48,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3338218a3717aa65d2b8d HTTP/1.1" 200 None 2026-03-27 00:59:48,286 - lava_callback - DEBUG - Processing node 69c3338218a3717aa65d2b8d: baseline-arm-baylibre 2026-03-27 00:59:48,286 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3338218a3717aa65d2b8d 2026-03-27 00:59:48,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:59:48,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3338218a3717aa65d2b8d HTTP/1.1" 200 None 2026-03-27 00:59:48,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:59:48,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-27 00:59:48,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:59:48,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3338218a3717aa65d2b8d HTTP/1.1" 200 None 2026-03-27 00:59:48,337 - lava_callback - ERROR - Error processing callback for node 69c3338218a3717aa65d2b8d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-27 00:59:54,096 - lava_callback - INFO - Received callback for node 69c5d5aa2cb62322ad96633e INFO: 172.20.0.1:41320 - "POST /node/69c5d5aa2cb62322ad96633e HTTP/1.0" 202 Accepted 2026-03-27 00:59:54,102 - lava_callback - INFO - Starting async job submit for node 69c5d5aa2cb62322ad96633e 2026-03-27 00:59:54,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:59:54,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5aa2cb62322ad96633e HTTP/1.1" 200 None 2026-03-27 00:59:54,140 - lava_callback - DEBUG - Processing node 69c5d5aa2cb62322ad96633e: baseline-arm64-broonie 2026-03-27 00:59:54,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:59:54,545 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:59:54,547 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c5d5aa2cb62322ad96633e/log.txt.gz 2026-03-27 00:59:54,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:59:54,904 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:59:54,906 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c5d5aa2cb62322ad96633e/lava-logs.yaml 2026-03-27 00:59:54,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 00:59:55,043 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 00:59:55,045 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c5d5aa2cb62322ad96633e/lava_callback.json.gz 2026-03-27 00:59:55,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:59:55,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5aa2cb62322ad96633e HTTP/1.1" 200 None 2026-03-27 00:59:55,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:59:55,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-27 00:59:55,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:59:55,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d5aa2cb62322ad96633e HTTP/1.1" 200 None 2026-03-27 00:59:55,125 - lava_callback - INFO - Completed processing callback for node 69c5d5aa2cb62322ad96633e INFO: 172.20.0.1:41328 - "GET / HTTP/1.0" 200 OK 2026-03-27 00:59:59,645 - lava_callback - INFO - Received callback for node 69c5d5d32cb62322ad96639a INFO: 172.20.0.1:41342 - "POST /node/69c5d5d32cb62322ad96639a HTTP/1.0" 202 Accepted 2026-03-27 00:59:59,653 - lava_callback - INFO - Starting async job submit for node 69c5d5d32cb62322ad96639a 2026-03-27 00:59:59,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 00:59:59,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5d32cb62322ad96639a HTTP/1.1" 200 None 2026-03-27 00:59:59,710 - lava_callback - DEBUG - Processing node 69c5d5d32cb62322ad96639a: baseline-arm64-foundriesio 2026-03-27 01:00:00,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:00:00,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-27 01:00:00,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 01:00:00,652 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:00:00,654 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c5d5d32cb62322ad96639a/log.txt.gz 2026-03-27 01:00:01,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 01:00:01,364 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:00:01,366 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c5d5d32cb62322ad96639a/lava-logs.yaml 2026-03-27 01:00:01,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 01:00:01,633 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:00:01,635 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c5d5d32cb62322ad96639a/lava_callback.json.gz DEBUG: 69c5d5d32cb62322ad96639a Transitting job node result: pass -> incomplete 2026-03-27 01:00:01,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:00:01,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5d32cb62322ad96639a HTTP/1.1" 200 None 2026-03-27 01:00:01,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:00:01,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-27 01:00:01,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:00:01,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d5d32cb62322ad96639a HTTP/1.1" 200 None 2026-03-27 01:00:01,741 - lava_callback - INFO - Completed processing callback for node 69c5d5d32cb62322ad96639a INFO: 172.20.0.1:49706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57326 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34946 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:00:28,283 - lava_callback - INFO - Received callback for node 69c333a718a3717aa65d2b99 INFO: 172.20.0.1:34952 - "POST /node/69c333a718a3717aa65d2b99 HTTP/1.0" 202 Accepted 2026-03-27 01:00:28,288 - lava_callback - INFO - Starting async job submit for node 69c333a718a3717aa65d2b99 2026-03-27 01:00:28,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:00:28,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c333a718a3717aa65d2b99 HTTP/1.1" 200 None 2026-03-27 01:00:28,310 - lava_callback - DEBUG - Processing node 69c333a718a3717aa65d2b99: baseline-arm-baylibre 2026-03-27 01:00:28,310 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c333a718a3717aa65d2b99 2026-03-27 01:00:28,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:00:28,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c333a718a3717aa65d2b99 HTTP/1.1" 200 None 2026-03-27 01:00:28,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:00:28,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-27 01:00:28,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:00:28,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c333a718a3717aa65d2b99 HTTP/1.1" 200 None 2026-03-27 01:00:28,374 - lava_callback - ERROR - Error processing callback for node 69c333a718a3717aa65d2b99: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-27 01:00:30,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:00:30,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:42802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40716 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:01:08,355 - lava_callback - INFO - Received callback for node 69c333cd18a3717aa65d2ba8 INFO: 172.20.0.1:40720 - "POST /node/69c333cd18a3717aa65d2ba8 HTTP/1.0" 202 Accepted 2026-03-27 01:01:08,360 - lava_callback - INFO - Starting async job submit for node 69c333cd18a3717aa65d2ba8 2026-03-27 01:01:08,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:01:08,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c333cd18a3717aa65d2ba8 HTTP/1.1" 200 None 2026-03-27 01:01:08,376 - lava_callback - DEBUG - Processing node 69c333cd18a3717aa65d2ba8: baseline-arm-baylibre 2026-03-27 01:01:08,376 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c333cd18a3717aa65d2ba8 2026-03-27 01:01:08,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:01:08,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c333cd18a3717aa65d2ba8 HTTP/1.1" 200 None 2026-03-27 01:01:08,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:01:08,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-27 01:01:08,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:01:08,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c333cd18a3717aa65d2ba8 HTTP/1.1" 200 None 2026-03-27 01:01:08,475 - lava_callback - ERROR - Error processing callback for node 69c333cd18a3717aa65d2ba8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57404 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57418 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:01:19,407 - lava_callback - INFO - Received callback for node 69c5d6cc2cb62322ad96646f INFO: 172.20.0.1:57428 - "POST /node/69c5d6cc2cb62322ad96646f HTTP/1.0" 202 Accepted 2026-03-27 01:01:19,413 - lava_callback - INFO - Starting async job submit for node 69c5d6cc2cb62322ad96646f 2026-03-27 01:01:19,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:01:19,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d6cc2cb62322ad96646f HTTP/1.1" 200 None 2026-03-27 01:01:19,432 - lava_callback - DEBUG - Processing node 69c5d6cc2cb62322ad96646f: baseline-arm 2026-03-27 01:01:19,437 - lava_callback - WARNING - Failed to upload log 2026-03-27 01:01:19,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 01:01:19,611 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:01:19,613 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c5d6cc2cb62322ad96646f/lava-logs.yaml 2026-03-27 01:01:19,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 01:01:19,768 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:01:19,770 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c5d6cc2cb62322ad96646f/lava_callback.json.gz 2026-03-27 01:01:19,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:01:19,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d6cc2cb62322ad96646f HTTP/1.1" 200 None 2026-03-27 01:01:19,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:01:19,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-27 01:01:19,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:01:19,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d6cc2cb62322ad96646f HTTP/1.1" 200 None 2026-03-27 01:01:19,835 - lava_callback - INFO - Completed processing callback for node 69c5d6cc2cb62322ad96646f 2026-03-27 01:01:24,285 - lava_callback - INFO - Received callback for node 69c5b89dabf66a8f4d9bcd1b INFO: 172.20.0.1:46334 - "POST /node/69c5b89dabf66a8f4d9bcd1b HTTP/1.0" 202 Accepted 2026-03-27 01:01:24,291 - lava_callback - INFO - Starting async job submit for node 69c5b89dabf66a8f4d9bcd1b 2026-03-27 01:01:24,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:01:24,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b89dabf66a8f4d9bcd1b HTTP/1.1" 200 None 2026-03-27 01:01:24,354 - lava_callback - DEBUG - Processing node 69c5b89dabf66a8f4d9bcd1b: baseline-arm64-broonie 2026-03-27 01:01:24,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 01:01:24,796 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:01:24,798 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c5b89dabf66a8f4d9bcd1b/log.txt.gz 2026-03-27 01:01:24,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 01:01:25,172 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:01:25,173 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c5b89dabf66a8f4d9bcd1b/lava-logs.yaml 2026-03-27 01:01:25,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 01:01:25,329 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:01:25,331 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c5b89dabf66a8f4d9bcd1b/lava_callback.json.gz 2026-03-27 01:01:25,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:01:25,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b89dabf66a8f4d9bcd1b HTTP/1.1" 200 None 2026-03-27 01:01:25,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:01:25,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-27 01:01:25,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:01:25,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5b89dabf66a8f4d9bcd1b HTTP/1.1" 200 None 2026-03-27 01:01:25,410 - lava_callback - INFO - Completed processing callback for node 69c5b89dabf66a8f4d9bcd1b INFO: 172.20.0.1:46342 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:01:30,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:01:30,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:58216 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:01:44,446 - lava_callback - INFO - Received callback for node 69c5d5b12cb62322ad966344 INFO: 172.20.0.1:39138 - "POST /node/69c5d5b12cb62322ad966344 HTTP/1.0" 202 Accepted 2026-03-27 01:01:44,452 - lava_callback - INFO - Starting async job submit for node 69c5d5b12cb62322ad966344 2026-03-27 01:01:44,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:01:44,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5b12cb62322ad966344 HTTP/1.1" 200 None 2026-03-27 01:01:44,501 - lava_callback - DEBUG - Processing node 69c5d5b12cb62322ad966344: baseline-arm64-broonie 2026-03-27 01:01:44,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 01:01:44,967 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:01:44,969 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c5d5b12cb62322ad966344/log.txt.gz 2026-03-27 01:01:45,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 01:01:45,479 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:01:45,481 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c5d5b12cb62322ad966344/lava-logs.yaml 2026-03-27 01:01:45,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 01:01:45,626 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:01:45,628 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c5d5b12cb62322ad966344/lava_callback.json.gz 2026-03-27 01:01:45,651 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:01:45,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5b12cb62322ad966344 HTTP/1.1" 200 None 2026-03-27 01:01:45,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:01:45,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-27 01:01:45,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:01:45,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d5b12cb62322ad966344 HTTP/1.1" 200 None 2026-03-27 01:01:45,707 - lava_callback - INFO - Completed processing callback for node 69c5d5b12cb62322ad966344 INFO: 172.20.0.1:39146 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:01:48,358 - lava_callback - INFO - Received callback for node 69c333f318a3717aa65d2bbb INFO: 172.20.0.1:39160 - "POST /node/69c333f318a3717aa65d2bbb HTTP/1.0" 202 Accepted 2026-03-27 01:01:48,363 - lava_callback - INFO - Starting async job submit for node 69c333f318a3717aa65d2bbb 2026-03-27 01:01:48,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:01:48,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c333f318a3717aa65d2bbb HTTP/1.1" 200 None 2026-03-27 01:01:48,379 - lava_callback - DEBUG - Processing node 69c333f318a3717aa65d2bbb: baseline-arm-baylibre 2026-03-27 01:01:48,379 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c333f318a3717aa65d2bbb 2026-03-27 01:01:48,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:01:48,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c333f318a3717aa65d2bbb HTTP/1.1" 200 None 2026-03-27 01:01:48,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:01:48,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-27 01:01:48,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:01:48,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c333f318a3717aa65d2bbb HTTP/1.1" 200 None 2026-03-27 01:01:48,434 - lava_callback - ERROR - Error processing callback for node 69c333f318a3717aa65d2bbb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41542 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:02:00,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:02:00,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:40234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50826 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50828 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:02:21,376 - lava_callback - INFO - Received callback for node 69c3341918a3717aa65d2bcc INFO: 172.20.0.1:54798 - "POST /node/69c3341918a3717aa65d2bcc HTTP/1.0" 202 Accepted 2026-03-27 01:02:21,382 - lava_callback - INFO - Starting async job submit for node 69c3341918a3717aa65d2bcc 2026-03-27 01:02:21,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:02:21,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3341918a3717aa65d2bcc HTTP/1.1" 200 None 2026-03-27 01:02:21,402 - lava_callback - DEBUG - Processing node 69c3341918a3717aa65d2bcc: baseline-arm-clabbe 2026-03-27 01:02:21,402 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3341918a3717aa65d2bcc 2026-03-27 01:02:21,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:02:21,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3341918a3717aa65d2bcc HTTP/1.1" 200 None 2026-03-27 01:02:21,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:02:21,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-27 01:02:21,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:02:21,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3341918a3717aa65d2bcc HTTP/1.1" 200 None 2026-03-27 01:02:21,471 - lava_callback - ERROR - Error processing callback for node 69c3341918a3717aa65d2bcc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59650 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:03:01,191 - lava_callback - INFO - Received callback for node 69c3343e18a3717aa65d2bdb INFO: 172.20.0.1:36154 - "POST /node/69c3343e18a3717aa65d2bdb HTTP/1.0" 202 Accepted 2026-03-27 01:03:01,198 - lava_callback - INFO - Starting async job submit for node 69c3343e18a3717aa65d2bdb 2026-03-27 01:03:01,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:03:01,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3343e18a3717aa65d2bdb HTTP/1.1" 200 None 2026-03-27 01:03:01,220 - lava_callback - DEBUG - Processing node 69c3343e18a3717aa65d2bdb: baseline-arm-clabbe 2026-03-27 01:03:01,220 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3343e18a3717aa65d2bdb 2026-03-27 01:03:01,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:03:01,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3343e18a3717aa65d2bdb HTTP/1.1" 200 None 2026-03-27 01:03:01,240 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:03:01,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-27 01:03:01,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:03:01,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3343e18a3717aa65d2bdb HTTP/1.1" 200 None 2026-03-27 01:03:01,284 - lava_callback - ERROR - Error processing callback for node 69c3343e18a3717aa65d2bdb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51568 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51578 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:03:21,200 - lava_callback - INFO - Received callback for node 69c5d5622cb62322ad96626b INFO: 172.20.0.1:43140 - "POST /node/69c5d5622cb62322ad96626b HTTP/1.0" 202 Accepted 2026-03-27 01:03:21,209 - lava_callback - INFO - Starting async job submit for node 69c5d5622cb62322ad96626b 2026-03-27 01:03:21,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:03:21,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5622cb62322ad96626b HTTP/1.1" 200 None 2026-03-27 01:03:21,285 - lava_callback - DEBUG - Processing node 69c5d5622cb62322ad96626b: baseline-arm64 2026-03-27 01:03:21,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 01:03:21,868 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:03:21,870 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5d5622cb62322ad96626b/log.txt.gz 2026-03-27 01:03:22,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 01:03:22,435 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:03:22,437 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5d5622cb62322ad96626b/lava-logs.yaml 2026-03-27 01:03:22,448 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 01:03:22,642 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:03:22,644 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5d5622cb62322ad96626b/lava_callback.json.gz DEBUG: Setup failed for 69c5d5622cb62322ad96626b. Transitting job node result: incomplete -> fail 2026-03-27 01:03:22,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:03:22,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5622cb62322ad96626b HTTP/1.1" 200 None 2026-03-27 01:03:22,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:03:22,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-27 01:03:22,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:03:22,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d5622cb62322ad96626b HTTP/1.1" 200 None 2026-03-27 01:03:22,887 - lava_callback - INFO - Completed processing callback for node 69c5d5622cb62322ad96626b INFO: 172.20.0.1:43154 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:03:30,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:03:30,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:32778 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:03:41,386 - lava_callback - INFO - Received callback for node 69c3346418a3717aa65d2bed INFO: 172.20.0.1:33154 - "POST /node/69c3346418a3717aa65d2bed HTTP/1.0" 202 Accepted 2026-03-27 01:03:41,393 - lava_callback - INFO - Starting async job submit for node 69c3346418a3717aa65d2bed 2026-03-27 01:03:41,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:03:41,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3346418a3717aa65d2bed HTTP/1.1" 200 None 2026-03-27 01:03:41,415 - lava_callback - DEBUG - Processing node 69c3346418a3717aa65d2bed: baseline-arm-clabbe 2026-03-27 01:03:41,415 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3346418a3717aa65d2bed 2026-03-27 01:03:41,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:03:41,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3346418a3717aa65d2bed HTTP/1.1" 200 None 2026-03-27 01:03:41,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:03:41,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-27 01:03:41,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:03:41,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3346418a3717aa65d2bed HTTP/1.1" 200 None 2026-03-27 01:03:41,476 - lava_callback - ERROR - Error processing callback for node 69c3346418a3717aa65d2bed: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46774 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:04:08,998 - lava_callback - INFO - Received callback for node 69c3348618a3717aa65d2bf8 INFO: 172.20.0.1:46776 - "POST /node/69c3348618a3717aa65d2bf8 HTTP/1.0" 202 Accepted 2026-03-27 01:04:09,005 - lava_callback - INFO - Starting async job submit for node 69c3348618a3717aa65d2bf8 2026-03-27 01:04:09,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:04:09,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3348618a3717aa65d2bf8 HTTP/1.1" 200 None 2026-03-27 01:04:09,029 - lava_callback - DEBUG - Processing node 69c3348618a3717aa65d2bf8: baseline-arm-baylibre 2026-03-27 01:04:09,029 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3348618a3717aa65d2bf8 2026-03-27 01:04:09,030 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:04:09,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3348618a3717aa65d2bf8 HTTP/1.1" 200 None 2026-03-27 01:04:09,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:04:09,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-27 01:04:09,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:04:09,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3348618a3717aa65d2bf8 HTTP/1.1" 200 None 2026-03-27 01:04:09,097 - lava_callback - ERROR - Error processing callback for node 69c3348618a3717aa65d2bf8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55790 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37098 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:04:58,631 - lava_callback - INFO - Received callback for node 69c5d5c22cb62322ad966379 INFO: 172.20.0.1:37110 - "POST /node/69c5d5c22cb62322ad966379 HTTP/1.0" 202 Accepted 2026-03-27 01:04:58,640 - lava_callback - INFO - Starting async job submit for node 69c5d5c22cb62322ad966379 2026-03-27 01:04:58,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:04:58,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5c22cb62322ad966379 HTTP/1.1" 200 None 2026-03-27 01:04:58,689 - lava_callback - DEBUG - Processing node 69c5d5c22cb62322ad966379: baseline-arm64-clabbe 2026-03-27 01:04:59,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 01:04:59,540 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:04:59,542 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69c5d5c22cb62322ad966379/log.txt.gz 2026-03-27 01:04:59,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 01:05:00,107 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:05:00,109 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69c5d5c22cb62322ad966379/lava-logs.yaml 2026-03-27 01:05:00,119 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 01:05:00,315 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:05:00,317 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69c5d5c22cb62322ad966379/lava_callback.json.gz 2026-03-27 01:05:00,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:05:00,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5c22cb62322ad966379 HTTP/1.1" 200 None 2026-03-27 01:05:00,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:05:00,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-27 01:05:00,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:05:00,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d5c22cb62322ad966379 HTTP/1.1" 200 None 2026-03-27 01:05:00,419 - lava_callback - INFO - Completed processing callback for node 69c5d5c22cb62322ad966379 2026-03-27 01:05:00,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:05:00,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-27 01:05:01,357 - lava_callback - INFO - Received callback for node 69c334ac18a3717aa65d2c02 INFO: 172.20.0.1:55384 - "POST /node/69c334ac18a3717aa65d2c02 HTTP/1.0" 202 Accepted 2026-03-27 01:05:01,364 - lava_callback - INFO - Starting async job submit for node 69c334ac18a3717aa65d2c02 2026-03-27 01:05:01,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:05:01,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c334ac18a3717aa65d2c02 HTTP/1.1" 200 None 2026-03-27 01:05:01,381 - lava_callback - DEBUG - Processing node 69c334ac18a3717aa65d2c02: baseline-arm-clabbe 2026-03-27 01:05:01,381 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c334ac18a3717aa65d2c02 2026-03-27 01:05:01,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:05:01,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c334ac18a3717aa65d2c02 HTTP/1.1" 200 None 2026-03-27 01:05:01,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:05:01,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-27 01:05:01,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:05:01,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c334ac18a3717aa65d2c02 HTTP/1.1" 200 None 2026-03-27 01:05:01,445 - lava_callback - ERROR - Error processing callback for node 69c334ac18a3717aa65d2c02: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34882 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60894 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:05:41,407 - lava_callback - INFO - Received callback for node 69c334d218a3717aa65d2c1e INFO: 172.20.0.1:40194 - "POST /node/69c334d218a3717aa65d2c1e HTTP/1.0" 202 Accepted 2026-03-27 01:05:41,412 - lava_callback - INFO - Starting async job submit for node 69c334d218a3717aa65d2c1e 2026-03-27 01:05:41,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:05:41,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c334d218a3717aa65d2c1e HTTP/1.1" 200 None 2026-03-27 01:05:41,428 - lava_callback - DEBUG - Processing node 69c334d218a3717aa65d2c1e: baseline-arm-clabbe 2026-03-27 01:05:41,428 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c334d218a3717aa65d2c1e 2026-03-27 01:05:41,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:05:41,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c334d218a3717aa65d2c1e HTTP/1.1" 200 None 2026-03-27 01:05:41,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:05:41,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-27 01:05:41,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:05:41,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c334d218a3717aa65d2c1e HTTP/1.1" 200 None 2026-03-27 01:05:41,492 - lava_callback - ERROR - Error processing callback for node 69c334d218a3717aa65d2c1e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43504 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46916 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46928 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:07:24,988 - lava_callback - INFO - Received callback for node 69c5d3a12cb62322ad96615b INFO: 172.20.0.1:36678 - "POST /node/69c5d3a12cb62322ad96615b HTTP/1.0" 202 Accepted 2026-03-27 01:07:24,996 - lava_callback - INFO - Starting async job submit for node 69c5d3a12cb62322ad96615b 2026-03-27 01:07:25,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:07:25,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d3a12cb62322ad96615b HTTP/1.1" 200 None 2026-03-27 01:07:25,057 - lava_callback - DEBUG - Processing node 69c5d3a12cb62322ad96615b: baseline-nfs-arm64 2026-03-27 01:07:26,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 01:07:26,379 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:07:26,380 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5d3a12cb62322ad96615b/log.txt.gz INFO: 172.20.0.1:36682 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:07:27,016 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 01:07:27,275 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:07:27,277 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5d3a12cb62322ad96615b/lava-logs.yaml 2026-03-27 01:07:27,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 01:07:27,489 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:07:27,491 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5d3a12cb62322ad96615b/lava_callback.json.gz DEBUG: Setup failed for 69c5d3a12cb62322ad96615b. Transitting job node result: incomplete -> fail 2026-03-27 01:07:27,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:07:27,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d3a12cb62322ad96615b HTTP/1.1" 200 None 2026-03-27 01:07:27,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:07:27,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-27 01:07:27,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:07:27,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d3a12cb62322ad96615b HTTP/1.1" 200 None 2026-03-27 01:07:27,738 - lava_callback - INFO - Completed processing callback for node 69c5d3a12cb62322ad96615b 2026-03-27 01:07:30,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:07:30,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:52290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49168 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39894 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49142 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54476 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37686 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:11:28,950 - lava_callback - INFO - Received callback for node 69c3362d18a3717aa65d2c44 INFO: 172.20.0.1:37690 - "POST /node/69c3362d18a3717aa65d2c44 HTTP/1.0" 202 Accepted 2026-03-27 01:11:28,955 - lava_callback - INFO - Starting async job submit for node 69c3362d18a3717aa65d2c44 2026-03-27 01:11:28,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:11:28,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3362d18a3717aa65d2c44 HTTP/1.1" 200 None 2026-03-27 01:11:28,971 - lava_callback - DEBUG - Processing node 69c3362d18a3717aa65d2c44: baseline-arm64-baylibre 2026-03-27 01:11:28,971 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3362d18a3717aa65d2c44 2026-03-27 01:11:28,971 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:11:28,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3362d18a3717aa65d2c44 HTTP/1.1" 200 None 2026-03-27 01:11:28,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:11:29,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-27 01:11:29,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:11:29,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3362d18a3717aa65d2c44 HTTP/1.1" 200 None 2026-03-27 01:11:29,033 - lava_callback - ERROR - Error processing callback for node 69c3362d18a3717aa65d2c44: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48834 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:11:40,543 - lava_callback - INFO - Received callback for node 69c5d9192cb62322ad968ac2 INFO: 172.20.0.1:53788 - "POST /node/69c5d9192cb62322ad968ac2 HTTP/1.0" 202 Accepted 2026-03-27 01:11:40,550 - lava_callback - INFO - Starting async job submit for node 69c5d9192cb62322ad968ac2 2026-03-27 01:11:40,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:11:40,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d9192cb62322ad968ac2 HTTP/1.1" 200 None 2026-03-27 01:11:40,598 - lava_callback - DEBUG - Processing node 69c5d9192cb62322ad968ac2: baseline-arm64 2026-03-27 01:11:40,766 - lava_callback - WARNING - Failed to upload log 2026-03-27 01:11:40,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 01:11:41,001 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:11:41,003 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5d9192cb62322ad968ac2/lava-logs.yaml 2026-03-27 01:11:41,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 01:11:41,149 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:11:41,150 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5d9192cb62322ad968ac2/lava_callback.json.gz 2026-03-27 01:11:41,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:11:41,195 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d9192cb62322ad968ac2 HTTP/1.1" 200 None 2026-03-27 01:11:41,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:11:41,210 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-27 01:11:41,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:11:41,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d9192cb62322ad968ac2 HTTP/1.1" 200 None 2026-03-27 01:11:41,290 - lava_callback - INFO - Completed processing callback for node 69c5d9192cb62322ad968ac2 INFO: 172.20.0.1:53794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37032 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:12:00,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:12:00,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:53436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36658 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53722 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:12:42,164 - lava_callback - INFO - Received callback for node 69c5d9782cb62322ad968ad4 INFO: 172.20.0.1:48146 - "POST /node/69c5d9782cb62322ad968ad4 HTTP/1.0" 202 Accepted 2026-03-27 01:12:42,170 - lava_callback - INFO - Starting async job submit for node 69c5d9782cb62322ad968ad4 2026-03-27 01:12:42,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:12:42,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d9782cb62322ad968ad4 HTTP/1.1" 200 None 2026-03-27 01:12:42,224 - lava_callback - DEBUG - Processing node 69c5d9782cb62322ad968ad4: baseline-nfs-arm64 2026-03-27 01:12:42,226 - lava_callback - WARNING - Failed to upload log 2026-03-27 01:12:42,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 01:12:42,369 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:12:42,370 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5d9782cb62322ad968ad4/lava-logs.yaml 2026-03-27 01:12:42,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 01:12:42,494 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:12:42,496 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5d9782cb62322ad968ad4/lava_callback.json.gz 2026-03-27 01:12:42,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:12:42,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d9782cb62322ad968ad4 HTTP/1.1" 200 None 2026-03-27 01:12:42,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:12:42,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-27 01:12:42,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:12:42,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d9782cb62322ad968ad4 HTTP/1.1" 200 None 2026-03-27 01:12:42,569 - lava_callback - INFO - Completed processing callback for node 69c5d9782cb62322ad968ad4 INFO: 172.20.0.1:48152 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:12:49,149 - lava_callback - INFO - Received callback for node 69c3367e18a3717aa65d2c51 INFO: 172.20.0.1:48154 - "POST /node/69c3367e18a3717aa65d2c51 HTTP/1.0" 202 Accepted 2026-03-27 01:12:49,156 - lava_callback - INFO - Starting async job submit for node 69c3367e18a3717aa65d2c51 2026-03-27 01:12:49,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:12:49,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3367e18a3717aa65d2c51 HTTP/1.1" 200 None 2026-03-27 01:12:49,180 - lava_callback - DEBUG - Processing node 69c3367e18a3717aa65d2c51: baseline-arm64-baylibre 2026-03-27 01:12:49,180 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3367e18a3717aa65d2c51 2026-03-27 01:12:49,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:12:49,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3367e18a3717aa65d2c51 HTTP/1.1" 200 None 2026-03-27 01:12:49,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:12:49,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-27 01:12:49,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:12:49,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3367e18a3717aa65d2c51 HTTP/1.1" 200 None 2026-03-27 01:12:49,245 - lava_callback - ERROR - Error processing callback for node 69c3367e18a3717aa65d2c51: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57140 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:13:00,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:13:00,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:55656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59656 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52050 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:14:09,068 - lava_callback - INFO - Received callback for node 69c336cf18a3717aa65d2c5e INFO: 172.20.0.1:52060 - "POST /node/69c336cf18a3717aa65d2c5e HTTP/1.0" 202 Accepted 2026-03-27 01:14:09,074 - lava_callback - INFO - Starting async job submit for node 69c336cf18a3717aa65d2c5e 2026-03-27 01:14:09,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:14:09,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c336cf18a3717aa65d2c5e HTTP/1.1" 200 None 2026-03-27 01:14:09,091 - lava_callback - DEBUG - Processing node 69c336cf18a3717aa65d2c5e: baseline-arm64-baylibre 2026-03-27 01:14:09,092 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c336cf18a3717aa65d2c5e 2026-03-27 01:14:09,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:14:09,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c336cf18a3717aa65d2c5e HTTP/1.1" 200 None 2026-03-27 01:14:09,107 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:14:09,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-27 01:14:09,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:14:09,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c336cf18a3717aa65d2c5e HTTP/1.1" 200 None 2026-03-27 01:14:09,144 - lava_callback - ERROR - Error processing callback for node 69c336cf18a3717aa65d2c5e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55430 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42278 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:14:39,619 - lava_callback - INFO - Received callback for node 69c5d5c02cb62322ad966375 INFO: 172.20.0.1:42284 - "POST /node/69c5d5c02cb62322ad966375 HTTP/1.0" 202 Accepted 2026-03-27 01:14:39,626 - lava_callback - INFO - Starting async job submit for node 69c5d5c02cb62322ad966375 2026-03-27 01:14:39,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:14:39,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5c02cb62322ad966375 HTTP/1.1" 200 None 2026-03-27 01:14:39,703 - lava_callback - DEBUG - Processing node 69c5d5c02cb62322ad966375: baseline-arm 2026-03-27 01:14:40,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 01:14:40,456 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:14:40,458 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c5d5c02cb62322ad966375/log.txt.gz 2026-03-27 01:14:40,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 01:14:41,057 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:14:41,060 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c5d5c02cb62322ad966375/lava-logs.yaml 2026-03-27 01:14:41,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 01:14:41,266 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:14:41,268 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c5d5c02cb62322ad966375/lava_callback.json.gz DEBUG: Setup failed for 69c5d5c02cb62322ad966375. Transitting job node result: incomplete -> fail 2026-03-27 01:14:41,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:14:41,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5c02cb62322ad966375 HTTP/1.1" 200 None 2026-03-27 01:14:41,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:14:41,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-27 01:14:41,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:14:41,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d5c02cb62322ad966375 HTTP/1.1" 200 None 2026-03-27 01:14:41,568 - lava_callback - INFO - Completed processing callback for node 69c5d5c02cb62322ad966375 INFO: 172.20.0.1:35722 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:14:48,864 - lava_callback - INFO - Received callback for node 69c5d9c52cb62322ad968ae3 INFO: 172.20.0.1:35724 - "POST /node/69c5d9c52cb62322ad968ae3 HTTP/1.0" 202 Accepted 2026-03-27 01:14:48,871 - lava_callback - INFO - Starting async job submit for node 69c5d9c52cb62322ad968ae3 2026-03-27 01:14:48,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:14:48,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d9c52cb62322ad968ae3 HTTP/1.1" 200 None 2026-03-27 01:14:48,915 - lava_callback - DEBUG - Processing node 69c5d9c52cb62322ad968ae3: baseline-arm64 2026-03-27 01:14:49,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 01:14:49,207 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:14:49,209 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5d9c52cb62322ad968ae3/log.txt.gz 2026-03-27 01:14:49,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 01:14:49,475 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:14:49,476 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5d9c52cb62322ad968ae3/lava-logs.yaml 2026-03-27 01:14:49,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 01:14:49,629 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:14:49,630 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5d9c52cb62322ad968ae3/lava_callback.json.gz 2026-03-27 01:14:49,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:14:49,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d9c52cb62322ad968ae3 HTTP/1.1" 200 None 2026-03-27 01:14:49,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:14:49,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-27 01:14:49,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:14:49,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d9c52cb62322ad968ae3 HTTP/1.1" 200 None 2026-03-27 01:14:49,785 - lava_callback - INFO - Completed processing callback for node 69c5d9c52cb62322ad968ae3 INFO: 172.20.0.1:35946 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:15:00,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:15:00,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-27 01:15:06,179 - lava_callback - INFO - Received callback for node 69c5d56d2cb62322ad96628a INFO: 172.20.0.1:47080 - "POST /node/69c5d56d2cb62322ad96628a HTTP/1.0" 202 Accepted 2026-03-27 01:15:06,187 - lava_callback - INFO - Starting async job submit for node 69c5d56d2cb62322ad96628a 2026-03-27 01:15:06,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:15:06,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d56d2cb62322ad96628a HTTP/1.1" 200 None 2026-03-27 01:15:06,246 - lava_callback - DEBUG - Processing node 69c5d56d2cb62322ad96628a: baseline-nfs-arm64 INFO: 172.20.0.1:47084 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:15:07,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 01:15:07,734 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:15:07,736 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5d56d2cb62322ad96628a/log.txt.gz 2026-03-27 01:15:08,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 01:15:08,600 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:15:08,602 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5d56d2cb62322ad96628a/lava-logs.yaml 2026-03-27 01:15:08,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 01:15:08,800 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:15:08,803 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5d56d2cb62322ad96628a/lava_callback.json.gz DEBUG: Setup failed for 69c5d56d2cb62322ad96628a. Transitting job node result: incomplete -> fail 2026-03-27 01:15:08,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:15:08,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d56d2cb62322ad96628a HTTP/1.1" 200 None 2026-03-27 01:15:08,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:15:08,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-27 01:15:08,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:15:08,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d56d2cb62322ad96628a HTTP/1.1" 200 None 2026-03-27 01:15:09,053 - lava_callback - INFO - Completed processing callback for node 69c5d56d2cb62322ad96628a INFO: 172.20.0.1:40424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40430 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60640 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:15:29,134 - lava_callback - INFO - Received callback for node 69c3372618a3717aa65d2c73 INFO: 172.20.0.1:60644 - "POST /node/69c3372618a3717aa65d2c73 HTTP/1.0" 202 Accepted 2026-03-27 01:15:29,140 - lava_callback - INFO - Starting async job submit for node 69c3372618a3717aa65d2c73 2026-03-27 01:15:29,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:15:29,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3372618a3717aa65d2c73 HTTP/1.1" 200 None 2026-03-27 01:15:29,156 - lava_callback - DEBUG - Processing node 69c3372618a3717aa65d2c73: baseline-arm64-baylibre 2026-03-27 01:15:29,156 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3372618a3717aa65d2c73 2026-03-27 01:15:29,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:15:29,170 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3372618a3717aa65d2c73 HTTP/1.1" 200 None 2026-03-27 01:15:29,171 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:15:29,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-27 01:15:29,186 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:15:29,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3372618a3717aa65d2c73 HTTP/1.1" 200 None 2026-03-27 01:15:29,215 - lava_callback - ERROR - Error processing callback for node 69c3372618a3717aa65d2c73: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-27 01:15:30,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:15:30,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:57076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41278 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37324 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48284 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:17:29,281 - lava_callback - INFO - Received callback for node 69c337a118a3717aa65d2ce7 INFO: 172.20.0.1:48286 - "POST /node/69c337a118a3717aa65d2ce7 HTTP/1.0" 202 Accepted 2026-03-27 01:17:29,287 - lava_callback - INFO - Starting async job submit for node 69c337a118a3717aa65d2ce7 2026-03-27 01:17:29,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:17:29,302 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337a118a3717aa65d2ce7 HTTP/1.1" 200 None 2026-03-27 01:17:29,303 - lava_callback - DEBUG - Processing node 69c337a118a3717aa65d2ce7: baseline-arm64-baylibre 2026-03-27 01:17:29,304 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c337a118a3717aa65d2ce7 2026-03-27 01:17:29,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:17:29,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337a118a3717aa65d2ce7 HTTP/1.1" 200 None 2026-03-27 01:17:29,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:17:29,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-27 01:17:29,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:17:29,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c337a118a3717aa65d2ce7 HTTP/1.1" 200 None 2026-03-27 01:17:29,387 - lava_callback - ERROR - Error processing callback for node 69c337a118a3717aa65d2ce7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33266 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:17:49,217 - lava_callback - INFO - Received callback for node 69c337b918a3717aa65d2d1b INFO: 172.20.0.1:33282 - "POST /node/69c337b918a3717aa65d2d1b HTTP/1.0" 202 Accepted 2026-03-27 01:17:49,223 - lava_callback - INFO - Starting async job submit for node 69c337b918a3717aa65d2d1b 2026-03-27 01:17:49,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:17:49,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337b918a3717aa65d2d1b HTTP/1.1" 200 None 2026-03-27 01:17:49,240 - lava_callback - DEBUG - Processing node 69c337b918a3717aa65d2d1b: baseline-arm64-baylibre 2026-03-27 01:17:49,240 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c337b918a3717aa65d2d1b 2026-03-27 01:17:49,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:17:49,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337b918a3717aa65d2d1b HTTP/1.1" 200 None 2026-03-27 01:17:49,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:17:49,286 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-27 01:17:49,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:17:49,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c337b918a3717aa65d2d1b HTTP/1.1" 200 None 2026-03-27 01:17:49,313 - lava_callback - ERROR - Error processing callback for node 69c337b918a3717aa65d2d1b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-27 01:17:49,873 - lava_callback - INFO - Received callback for node 69c337ba18a3717aa65d2d1e INFO: 172.20.0.1:49062 - "POST /node/69c337ba18a3717aa65d2d1e HTTP/1.0" 202 Accepted 2026-03-27 01:17:49,879 - lava_callback - INFO - Starting async job submit for node 69c337ba18a3717aa65d2d1e 2026-03-27 01:17:49,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:17:49,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337ba18a3717aa65d2d1e HTTP/1.1" 200 None 2026-03-27 01:17:49,932 - lava_callback - DEBUG - Processing node 69c337ba18a3717aa65d2d1e: baseline-arm64-baylibre 2026-03-27 01:17:49,932 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c337ba18a3717aa65d2d1e 2026-03-27 01:17:49,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:17:49,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337ba18a3717aa65d2d1e HTTP/1.1" 200 None 2026-03-27 01:17:49,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:17:49,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-27 01:17:49,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:17:49,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c337ba18a3717aa65d2d1e HTTP/1.1" 200 None 2026-03-27 01:17:49,990 - lava_callback - ERROR - Error processing callback for node 69c337ba18a3717aa65d2d1e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-27 01:17:50,533 - lava_callback - INFO - Received callback for node 69c337bb18a3717aa65d2d3c INFO: 172.20.0.1:49068 - "POST /node/69c337bb18a3717aa65d2d3c HTTP/1.0" 202 Accepted 2026-03-27 01:17:50,538 - lava_callback - INFO - Starting async job submit for node 69c337bb18a3717aa65d2d3c 2026-03-27 01:17:50,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:17:50,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337bb18a3717aa65d2d3c HTTP/1.1" 200 None 2026-03-27 01:17:50,554 - lava_callback - DEBUG - Processing node 69c337bb18a3717aa65d2d3c: baseline-arm64-baylibre 2026-03-27 01:17:50,554 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c337bb18a3717aa65d2d3c 2026-03-27 01:17:50,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:17:50,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337bb18a3717aa65d2d3c HTTP/1.1" 200 None 2026-03-27 01:17:50,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:17:50,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-27 01:17:50,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:17:50,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c337bb18a3717aa65d2d3c HTTP/1.1" 200 None 2026-03-27 01:17:50,610 - lava_callback - ERROR - Error processing callback for node 69c337bb18a3717aa65d2d3c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53166 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:18:09,210 - lava_callback - INFO - Received callback for node 69c337be18a3717aa65d2d44 INFO: 172.20.0.1:53170 - "POST /node/69c337be18a3717aa65d2d44 HTTP/1.0" 202 Accepted 2026-03-27 01:18:09,217 - lava_callback - INFO - Starting async job submit for node 69c337be18a3717aa65d2d44 2026-03-27 01:18:09,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:18:09,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337be18a3717aa65d2d44 HTTP/1.1" 200 None 2026-03-27 01:18:09,235 - lava_callback - DEBUG - Processing node 69c337be18a3717aa65d2d44: baseline-arm64-baylibre 2026-03-27 01:18:09,235 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c337be18a3717aa65d2d44 2026-03-27 01:18:09,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:18:09,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337be18a3717aa65d2d44 HTTP/1.1" 200 None 2026-03-27 01:18:09,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:18:09,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-27 01:18:09,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:18:09,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c337be18a3717aa65d2d44 HTTP/1.1" 200 None 2026-03-27 01:18:09,305 - lava_callback - ERROR - Error processing callback for node 69c337be18a3717aa65d2d44: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-27 01:18:09,888 - lava_callback - INFO - Received callback for node 69c337bf18a3717aa65d2d48 INFO: 172.20.0.1:58642 - "POST /node/69c337bf18a3717aa65d2d48 HTTP/1.0" 202 Accepted 2026-03-27 01:18:09,894 - lava_callback - INFO - Starting async job submit for node 69c337bf18a3717aa65d2d48 2026-03-27 01:18:09,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:18:09,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337bf18a3717aa65d2d48 HTTP/1.1" 200 None 2026-03-27 01:18:09,909 - lava_callback - DEBUG - Processing node 69c337bf18a3717aa65d2d48: baseline-arm64-baylibre 2026-03-27 01:18:09,910 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c337bf18a3717aa65d2d48 2026-03-27 01:18:09,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:18:09,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337bf18a3717aa65d2d48 HTTP/1.1" 200 None 2026-03-27 01:18:09,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:18:09,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-27 01:18:09,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:18:09,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c337bf18a3717aa65d2d48 HTTP/1.1" 200 None 2026-03-27 01:18:09,968 - lava_callback - ERROR - Error processing callback for node 69c337bf18a3717aa65d2d48: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-27 01:18:10,545 - lava_callback - INFO - Received callback for node 69c337c018a3717aa65d2d4b INFO: 172.20.0.1:58650 - "POST /node/69c337c018a3717aa65d2d4b HTTP/1.0" 202 Accepted 2026-03-27 01:18:10,550 - lava_callback - INFO - Starting async job submit for node 69c337c018a3717aa65d2d4b 2026-03-27 01:18:10,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:18:10,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337c018a3717aa65d2d4b HTTP/1.1" 200 None 2026-03-27 01:18:10,567 - lava_callback - DEBUG - Processing node 69c337c018a3717aa65d2d4b: baseline-arm64-baylibre 2026-03-27 01:18:10,567 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c337c018a3717aa65d2d4b 2026-03-27 01:18:10,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:18:10,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337c018a3717aa65d2d4b HTTP/1.1" 200 None 2026-03-27 01:18:10,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:18:10,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-27 01:18:10,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:18:10,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c337c018a3717aa65d2d4b HTTP/1.1" 200 None 2026-03-27 01:18:10,618 - lava_callback - ERROR - Error processing callback for node 69c337c018a3717aa65d2d4b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58670 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58676 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:18:22,048 - lava_callback - INFO - Received callback for node 69c337d118a3717aa65d2d74 INFO: 172.20.0.1:33120 - "POST /node/69c337d118a3717aa65d2d74 HTTP/1.0" 202 Accepted 2026-03-27 01:18:22,054 - lava_callback - INFO - Starting async job submit for node 69c337d118a3717aa65d2d74 2026-03-27 01:18:22,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:18:22,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337d118a3717aa65d2d74 HTTP/1.1" 200 None 2026-03-27 01:18:22,071 - lava_callback - DEBUG - Processing node 69c337d118a3717aa65d2d74: baseline-arm64-clabbe 2026-03-27 01:18:22,071 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c337d118a3717aa65d2d74 2026-03-27 01:18:22,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:18:22,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337d118a3717aa65d2d74 HTTP/1.1" 200 None 2026-03-27 01:18:22,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:18:22,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-27 01:18:22,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:18:22,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c337d118a3717aa65d2d74 HTTP/1.1" 200 None 2026-03-27 01:18:22,126 - lava_callback - ERROR - Error processing callback for node 69c337d118a3717aa65d2d74: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-27 01:18:22,718 - lava_callback - INFO - Received callback for node 69c337d318a3717aa65d2d77 INFO: 172.20.0.1:33124 - "POST /node/69c337d318a3717aa65d2d77 HTTP/1.0" 202 Accepted 2026-03-27 01:18:22,724 - lava_callback - INFO - Starting async job submit for node 69c337d318a3717aa65d2d77 2026-03-27 01:18:22,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:18:22,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337d318a3717aa65d2d77 HTTP/1.1" 200 None 2026-03-27 01:18:22,740 - lava_callback - DEBUG - Processing node 69c337d318a3717aa65d2d77: baseline-arm64-clabbe 2026-03-27 01:18:22,740 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c337d318a3717aa65d2d77 2026-03-27 01:18:22,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:18:22,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337d318a3717aa65d2d77 HTTP/1.1" 200 None 2026-03-27 01:18:22,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:18:22,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-27 01:18:22,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:18:22,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c337d318a3717aa65d2d77 HTTP/1.1" 200 None 2026-03-27 01:18:22,792 - lava_callback - ERROR - Error processing callback for node 69c337d318a3717aa65d2d77: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-27 01:18:23,391 - lava_callback - INFO - Received callback for node 69c337d418a3717aa65d2d7a INFO: 172.20.0.1:33140 - "POST /node/69c337d418a3717aa65d2d7a HTTP/1.0" 202 Accepted 2026-03-27 01:18:23,396 - lava_callback - INFO - Starting async job submit for node 69c337d418a3717aa65d2d7a 2026-03-27 01:18:23,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:18:23,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337d418a3717aa65d2d7a HTTP/1.1" 200 None 2026-03-27 01:18:23,416 - lava_callback - DEBUG - Processing node 69c337d418a3717aa65d2d7a: baseline-arm64-clabbe 2026-03-27 01:18:23,417 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c337d418a3717aa65d2d7a 2026-03-27 01:18:23,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:18:23,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337d418a3717aa65d2d7a HTTP/1.1" 200 None 2026-03-27 01:18:23,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:18:23,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-27 01:18:23,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:18:23,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c337d418a3717aa65d2d7a HTTP/1.1" 200 None 2026-03-27 01:18:23,473 - lava_callback - ERROR - Error processing callback for node 69c337d418a3717aa65d2d7a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-27 01:18:24,074 - lava_callback - INFO - Received callback for node 69c337d518a3717aa65d2d7d INFO: 172.20.0.1:33156 - "POST /node/69c337d518a3717aa65d2d7d HTTP/1.0" 202 Accepted 2026-03-27 01:18:24,080 - lava_callback - INFO - Starting async job submit for node 69c337d518a3717aa65d2d7d 2026-03-27 01:18:24,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:18:24,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337d518a3717aa65d2d7d HTTP/1.1" 200 None 2026-03-27 01:18:24,096 - lava_callback - DEBUG - Processing node 69c337d518a3717aa65d2d7d: baseline-arm64-clabbe 2026-03-27 01:18:24,096 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c337d518a3717aa65d2d7d 2026-03-27 01:18:24,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:18:24,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337d518a3717aa65d2d7d HTTP/1.1" 200 None 2026-03-27 01:18:24,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:18:24,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-27 01:18:24,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:18:24,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c337d518a3717aa65d2d7d HTTP/1.1" 200 None 2026-03-27 01:18:24,163 - lava_callback - ERROR - Error processing callback for node 69c337d518a3717aa65d2d7d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-27 01:18:24,753 - lava_callback - INFO - Received callback for node 69c337d718a3717aa65d2d83 INFO: 172.20.0.1:33166 - "POST /node/69c337d718a3717aa65d2d83 HTTP/1.0" 202 Accepted 2026-03-27 01:18:24,759 - lava_callback - INFO - Starting async job submit for node 69c337d718a3717aa65d2d83 2026-03-27 01:18:24,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:18:24,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337d718a3717aa65d2d83 HTTP/1.1" 200 None 2026-03-27 01:18:24,775 - lava_callback - DEBUG - Processing node 69c337d718a3717aa65d2d83: baseline-arm64-clabbe 2026-03-27 01:18:24,775 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c337d718a3717aa65d2d83 2026-03-27 01:18:24,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:18:24,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337d718a3717aa65d2d83 HTTP/1.1" 200 None 2026-03-27 01:18:24,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:18:24,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-27 01:18:24,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:18:24,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c337d718a3717aa65d2d83 HTTP/1.1" 200 None 2026-03-27 01:18:24,841 - lava_callback - ERROR - Error processing callback for node 69c337d718a3717aa65d2d83: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-27 01:18:25,429 - lava_callback - INFO - Received callback for node 69c337d818a3717aa65d2d86 INFO: 172.20.0.1:33170 - "POST /node/69c337d818a3717aa65d2d86 HTTP/1.0" 202 Accepted 2026-03-27 01:18:25,435 - lava_callback - INFO - Starting async job submit for node 69c337d818a3717aa65d2d86 2026-03-27 01:18:25,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:18:25,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337d818a3717aa65d2d86 HTTP/1.1" 200 None 2026-03-27 01:18:25,450 - lava_callback - DEBUG - Processing node 69c337d818a3717aa65d2d86: baseline-arm64-clabbe 2026-03-27 01:18:25,451 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c337d818a3717aa65d2d86 2026-03-27 01:18:25,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:18:25,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337d818a3717aa65d2d86 HTTP/1.1" 200 None 2026-03-27 01:18:25,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:18:25,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-27 01:18:25,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:18:25,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c337d818a3717aa65d2d86 HTTP/1.1" 200 None 2026-03-27 01:18:25,515 - lava_callback - ERROR - Error processing callback for node 69c337d818a3717aa65d2d86: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47026 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:18:49,294 - lava_callback - INFO - Received callback for node 69c337f418a3717aa65d2dd2 INFO: 172.20.0.1:47036 - "POST /node/69c337f418a3717aa65d2dd2 HTTP/1.0" 202 Accepted 2026-03-27 01:18:49,300 - lava_callback - INFO - Starting async job submit for node 69c337f418a3717aa65d2dd2 2026-03-27 01:18:49,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:18:49,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337f418a3717aa65d2dd2 HTTP/1.1" 200 None 2026-03-27 01:18:49,316 - lava_callback - DEBUG - Processing node 69c337f418a3717aa65d2dd2: baseline-arm64-baylibre 2026-03-27 01:18:49,316 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c337f418a3717aa65d2dd2 2026-03-27 01:18:49,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:18:49,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337f418a3717aa65d2dd2 HTTP/1.1" 200 None 2026-03-27 01:18:49,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:18:49,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-27 01:18:49,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:18:49,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c337f418a3717aa65d2dd2 HTTP/1.1" 200 None 2026-03-27 01:18:49,375 - lava_callback - ERROR - Error processing callback for node 69c337f418a3717aa65d2dd2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55748 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:19:10,631 - lava_callback - INFO - Received callback for node 69c5da582cb62322ad968b0e INFO: 172.20.0.1:50400 - "POST /node/69c5da582cb62322ad968b0e HTTP/1.0" 202 Accepted 2026-03-27 01:19:10,639 - lava_callback - INFO - Starting async job submit for node 69c5da582cb62322ad968b0e 2026-03-27 01:19:10,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:19:10,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5da582cb62322ad968b0e HTTP/1.1" 200 None 2026-03-27 01:19:10,695 - lava_callback - DEBUG - Processing node 69c5da582cb62322ad968b0e: baseline-arm64-foundriesio 2026-03-27 01:19:11,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 01:19:11,651 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:19:11,652 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c5da582cb62322ad968b0e/log.txt.gz 2026-03-27 01:19:12,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 01:19:12,367 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:19:12,370 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c5da582cb62322ad968b0e/lava-logs.yaml 2026-03-27 01:19:12,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 01:19:12,581 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:19:12,583 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c5da582cb62322ad968b0e/lava_callback.json.gz DEBUG: 69c5da582cb62322ad968b0e Transitting job node result: pass -> incomplete 2026-03-27 01:19:12,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:19:12,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5da582cb62322ad968b0e HTTP/1.1" 200 None 2026-03-27 01:19:12,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:19:12,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-27 01:19:12,659 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:19:12,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5da582cb62322ad968b0e HTTP/1.1" 200 None 2026-03-27 01:19:12,686 - lava_callback - INFO - Completed processing callback for node 69c5da582cb62322ad968b0e INFO: 172.20.0.1:50410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50420 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59904 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:19:30,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:19:30,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:37288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53432 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:19:54,956 - lava_callback - INFO - Received callback for node 69c5d7182cb62322ad968a29 INFO: 172.20.0.1:44226 - "POST /node/69c5d7182cb62322ad968a29 HTTP/1.0" 202 Accepted 2026-03-27 01:19:54,963 - lava_callback - INFO - Starting async job submit for node 69c5d7182cb62322ad968a29 2026-03-27 01:19:55,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:19:55,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d7182cb62322ad968a29 HTTP/1.1" 200 None 2026-03-27 01:19:55,043 - lava_callback - DEBUG - Processing node 69c5d7182cb62322ad968a29: ltp-smoke 2026-03-27 01:19:55,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 01:19:55,678 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:19:55,679 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c5d7182cb62322ad968a29/log.txt.gz 2026-03-27 01:19:55,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 01:19:56,175 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:19:56,177 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c5d7182cb62322ad968a29/lava-logs.yaml 2026-03-27 01:19:56,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 01:19:56,348 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:19:56,350 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c5d7182cb62322ad968a29/lava_callback.json.gz 2026-03-27 01:19:56,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:19:56,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d7182cb62322ad968a29 HTTP/1.1" 200 None INFO: 172.20.0.1:44240 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:19:56,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:19:56,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc312cb62322ad965dad HTTP/1.1" 200 None 2026-03-27 01:19:56,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:19:56,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d7182cb62322ad968a29 HTTP/1.1" 200 None 2026-03-27 01:19:56,709 - lava_callback - INFO - Completed processing callback for node 69c5d7182cb62322ad968a29 2026-03-27 01:20:00,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:20:00,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:48494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48406 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37800 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55028 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55038 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:22:21,852 - lava_callback - INFO - Received callback for node 69c5db922cb62322ad968b56 INFO: 172.20.0.1:41260 - "POST /node/69c5db922cb62322ad968b56 HTTP/1.0" 202 Accepted 2026-03-27 01:22:21,858 - lava_callback - INFO - Starting async job submit for node 69c5db922cb62322ad968b56 2026-03-27 01:22:21,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:22:21,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5db922cb62322ad968b56 HTTP/1.1" 200 None 2026-03-27 01:22:21,939 - lava_callback - DEBUG - Processing node 69c5db922cb62322ad968b56: baseline-arm64 2026-03-27 01:22:22,076 - lava_callback - WARNING - Failed to upload log 2026-03-27 01:22:22,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 01:22:22,318 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:22:22,320 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5db922cb62322ad968b56/lava-logs.yaml 2026-03-27 01:22:22,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 01:22:22,457 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:22:22,458 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5db922cb62322ad968b56/lava_callback.json.gz 2026-03-27 01:22:22,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:22:22,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5db922cb62322ad968b56 HTTP/1.1" 200 None 2026-03-27 01:22:22,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:22:22,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-27 01:22:22,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:22:22,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5db922cb62322ad968b56 HTTP/1.1" 200 None 2026-03-27 01:22:22,609 - lava_callback - INFO - Completed processing callback for node 69c5db922cb62322ad968b56 INFO: 172.20.0.1:41274 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:22:30,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:22:30,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:35098 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:22:40,976 - lava_callback - INFO - Received callback for node 69c5da992cb62322ad968b13 INFO: 172.20.0.1:60708 - "POST /node/69c5da992cb62322ad968b13 HTTP/1.0" 202 Accepted 2026-03-27 01:22:40,983 - lava_callback - INFO - Starting async job submit for node 69c5da992cb62322ad968b13 2026-03-27 01:22:41,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:22:41,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5da992cb62322ad968b13 HTTP/1.1" 200 None 2026-03-27 01:22:41,041 - lava_callback - DEBUG - Processing node 69c5da992cb62322ad968b13: baseline-arm64 2026-03-27 01:22:41,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 01:22:41,550 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:22:41,552 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5da992cb62322ad968b13/log.txt.gz 2026-03-27 01:22:41,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 01:22:42,004 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:22:42,006 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5da992cb62322ad968b13/lava-logs.yaml 2026-03-27 01:22:42,015 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 01:22:42,179 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:22:42,181 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5da992cb62322ad968b13/lava_callback.json.gz DEBUG: Setup failed for 69c5da992cb62322ad968b13. Transitting job node result: incomplete -> fail 2026-03-27 01:22:42,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:22:42,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5da992cb62322ad968b13 HTTP/1.1" 200 None 2026-03-27 01:22:42,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:22:42,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-27 01:22:42,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:22:42,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5da992cb62322ad968b13 HTTP/1.1" 200 None 2026-03-27 01:22:42,401 - lava_callback - INFO - Completed processing callback for node 69c5da992cb62322ad968b13 INFO: 172.20.0.1:60714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35076 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:23:00,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:23:00,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:42714 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:23:09,891 - lava_callback - INFO - Received callback for node 69c5dbef2cb62322ad968b67 INFO: 172.20.0.1:53966 - "POST /node/69c5dbef2cb62322ad968b67 HTTP/1.0" 202 Accepted 2026-03-27 01:23:09,897 - lava_callback - INFO - Starting async job submit for node 69c5dbef2cb62322ad968b67 2026-03-27 01:23:09,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:23:09,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5dbef2cb62322ad968b67 HTTP/1.1" 200 None 2026-03-27 01:23:09,914 - lava_callback - DEBUG - Processing node 69c5dbef2cb62322ad968b67: baseline-nfs-arm64 2026-03-27 01:23:09,916 - lava_callback - WARNING - Failed to upload log 2026-03-27 01:23:09,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 01:23:10,243 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:23:10,245 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5dbef2cb62322ad968b67/lava-logs.yaml 2026-03-27 01:23:10,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 01:23:10,409 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:23:10,411 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5dbef2cb62322ad968b67/lava_callback.json.gz 2026-03-27 01:23:10,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:23:10,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5dbef2cb62322ad968b67 HTTP/1.1" 200 None 2026-03-27 01:23:10,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:23:10,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-27 01:23:10,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:23:10,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5dbef2cb62322ad968b67 HTTP/1.1" 200 None 2026-03-27 01:23:10,492 - lava_callback - INFO - Completed processing callback for node 69c5dbef2cb62322ad968b67 INFO: 172.20.0.1:53978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53992 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43904 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:23:30,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:23:30,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:58672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34042 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34056 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:24:22,225 - lava_callback - INFO - Received callback for node 69c3394018a3717aa65d2ed3 INFO: 172.20.0.1:42166 - "POST /node/69c3394018a3717aa65d2ed3 HTTP/1.0" 202 Accepted 2026-03-27 01:24:22,231 - lava_callback - INFO - Starting async job submit for node 69c3394018a3717aa65d2ed3 2026-03-27 01:24:22,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:24:22,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3394018a3717aa65d2ed3 HTTP/1.1" 200 None 2026-03-27 01:24:22,247 - lava_callback - DEBUG - Processing node 69c3394018a3717aa65d2ed3: baseline-arm64-clabbe 2026-03-27 01:24:22,247 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3394018a3717aa65d2ed3 2026-03-27 01:24:22,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:24:22,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3394018a3717aa65d2ed3 HTTP/1.1" 200 None 2026-03-27 01:24:22,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:24:22,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-27 01:24:22,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:24:22,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3394018a3717aa65d2ed3 HTTP/1.1" 200 None 2026-03-27 01:24:22,305 - lava_callback - ERROR - Error processing callback for node 69c3394018a3717aa65d2ed3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33708 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41694 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:25:49,523 - lava_callback - INFO - Received callback for node 69c5dc602cb62322ad968b7f INFO: 172.20.0.1:41700 - "POST /node/69c5dc602cb62322ad968b7f HTTP/1.0" 202 Accepted 2026-03-27 01:25:49,530 - lava_callback - INFO - Starting async job submit for node 69c5dc602cb62322ad968b7f 2026-03-27 01:25:49,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:25:49,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5dc602cb62322ad968b7f HTTP/1.1" 200 None 2026-03-27 01:25:49,592 - lava_callback - DEBUG - Processing node 69c5dc602cb62322ad968b7f: baseline-arm64 2026-03-27 01:25:49,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 01:25:49,935 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:25:49,936 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5dc602cb62322ad968b7f/log.txt.gz 2026-03-27 01:25:50,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 01:25:50,241 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:25:50,243 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5dc602cb62322ad968b7f/lava-logs.yaml 2026-03-27 01:25:50,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 01:25:50,459 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:25:50,461 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5dc602cb62322ad968b7f/lava_callback.json.gz 2026-03-27 01:25:50,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:25:50,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5dc602cb62322ad968b7f HTTP/1.1" 200 None 2026-03-27 01:25:50,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:25:50,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-27 01:25:50,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:25:50,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5dc602cb62322ad968b7f HTTP/1.1" 200 None 2026-03-27 01:25:50,629 - lava_callback - INFO - Completed processing callback for node 69c5dc602cb62322ad968b7f INFO: 172.20.0.1:52782 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:26:00,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:26:00,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:54120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42914 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42274 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45094 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56954 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48064 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:29:42,422 - lava_callback - INFO - Received callback for node 69c33a8218a3717aa65d2f19 INFO: 172.20.0.1:51678 - "POST /node/69c33a8218a3717aa65d2f19 HTTP/1.0" 202 Accepted 2026-03-27 01:29:42,428 - lava_callback - INFO - Starting async job submit for node 69c33a8218a3717aa65d2f19 2026-03-27 01:29:42,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:29:42,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c33a8218a3717aa65d2f19 HTTP/1.1" 200 None 2026-03-27 01:29:42,443 - lava_callback - DEBUG - Processing node 69c33a8218a3717aa65d2f19: baseline-arm64-clabbe 2026-03-27 01:29:42,443 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c33a8218a3717aa65d2f19 2026-03-27 01:29:42,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:29:42,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c33a8218a3717aa65d2f19 HTTP/1.1" 200 None 2026-03-27 01:29:42,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:29:42,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-27 01:29:42,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:29:42,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c33a8218a3717aa65d2f19 HTTP/1.1" 200 None 2026-03-27 01:29:42,519 - lava_callback - ERROR - Error processing callback for node 69c33a8218a3717aa65d2f19: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60062 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52586 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37830 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48874 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51792 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56516 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56526 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:35:22,684 - lava_callback - INFO - Received callback for node 69c33bc818a3717aa65d2f66 INFO: 172.20.0.1:46120 - "POST /node/69c33bc818a3717aa65d2f66 HTTP/1.0" 202 Accepted 2026-03-27 01:35:22,689 - lava_callback - INFO - Starting async job submit for node 69c33bc818a3717aa65d2f66 2026-03-27 01:35:22,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:35:22,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c33bc818a3717aa65d2f66 HTTP/1.1" 200 None 2026-03-27 01:35:22,704 - lava_callback - DEBUG - Processing node 69c33bc818a3717aa65d2f66: baseline-arm64-clabbe 2026-03-27 01:35:22,704 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c33bc818a3717aa65d2f66 2026-03-27 01:35:22,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:35:22,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c33bc818a3717aa65d2f66 HTTP/1.1" 200 None 2026-03-27 01:35:22,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:35:22,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-27 01:35:22,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:35:22,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c33bc818a3717aa65d2f66 HTTP/1.1" 200 None 2026-03-27 01:35:22,788 - lava_callback - ERROR - Error processing callback for node 69c33bc818a3717aa65d2f66: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40360 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49500 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56744 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:38:02,862 - lava_callback - INFO - Received callback for node 69c33c6918a3717aa65d2f92 INFO: 172.20.0.1:57812 - "POST /node/69c33c6918a3717aa65d2f92 HTTP/1.0" 202 Accepted 2026-03-27 01:38:02,868 - lava_callback - INFO - Starting async job submit for node 69c33c6918a3717aa65d2f92 2026-03-27 01:38:02,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:38:02,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c33c6918a3717aa65d2f92 HTTP/1.1" 200 None 2026-03-27 01:38:02,884 - lava_callback - DEBUG - Processing node 69c33c6918a3717aa65d2f92: baseline-arm64-clabbe 2026-03-27 01:38:02,884 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c33c6918a3717aa65d2f92 2026-03-27 01:38:02,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:38:02,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c33c6918a3717aa65d2f92 HTTP/1.1" 200 None 2026-03-27 01:38:02,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:38:02,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-27 01:38:02,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:38:02,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c33c6918a3717aa65d2f92 HTTP/1.1" 200 None 2026-03-27 01:38:02,937 - lava_callback - ERROR - Error processing callback for node 69c33c6918a3717aa65d2f92: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50980 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40312 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:38:45,295 - lava_callback - INFO - Received callback for node 69c5daf02cb62322ad968b35 INFO: 172.20.0.1:45156 - "POST /node/69c5daf02cb62322ad968b35 HTTP/1.0" 202 Accepted 2026-03-27 01:38:45,307 - lava_callback - INFO - Starting async job submit for node 69c5daf02cb62322ad968b35 2026-03-27 01:38:45,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:38:45,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5daf02cb62322ad968b35 HTTP/1.1" 200 None 2026-03-27 01:38:45,367 - lava_callback - DEBUG - Processing node 69c5daf02cb62322ad968b35: baseline-nfs-arm64 2026-03-27 01:38:46,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:45170 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:38:46,690 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:38:46,692 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5daf02cb62322ad968b35/log.txt.gz 2026-03-27 01:38:47,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 01:38:47,593 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:38:47,596 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5daf02cb62322ad968b35/lava-logs.yaml 2026-03-27 01:38:47,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 01:38:47,840 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:38:47,842 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5daf02cb62322ad968b35/lava_callback.json.gz DEBUG: Setup failed for 69c5daf02cb62322ad968b35. Transitting job node result: incomplete -> fail 2026-03-27 01:38:47,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:38:47,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5daf02cb62322ad968b35 HTTP/1.1" 200 None 2026-03-27 01:38:47,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:38:47,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-27 01:38:47,971 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:38:47,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5daf02cb62322ad968b35 HTTP/1.1" 200 None 2026-03-27 01:38:48,090 - lava_callback - INFO - Completed processing callback for node 69c5daf02cb62322ad968b35 INFO: 172.20.0.1:60910 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:39:00,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:39:00,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:41176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56820 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41908 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60590 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:40:42,922 - lava_callback - INFO - Received callback for node 69c33d1218a3717aa65d2fb6 INFO: 172.20.0.1:49492 - "POST /node/69c33d1218a3717aa65d2fb6 HTTP/1.0" 202 Accepted 2026-03-27 01:40:42,928 - lava_callback - INFO - Starting async job submit for node 69c33d1218a3717aa65d2fb6 2026-03-27 01:40:42,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:40:42,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c33d1218a3717aa65d2fb6 HTTP/1.1" 200 None 2026-03-27 01:40:42,945 - lava_callback - DEBUG - Processing node 69c33d1218a3717aa65d2fb6: baseline-arm64-clabbe 2026-03-27 01:40:42,945 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c33d1218a3717aa65d2fb6 2026-03-27 01:40:42,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:40:42,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c33d1218a3717aa65d2fb6 HTTP/1.1" 200 None 2026-03-27 01:40:42,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:40:42,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-27 01:40:42,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:40:42,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c33d1218a3717aa65d2fb6 HTTP/1.1" 200 None 2026-03-27 01:40:42,999 - lava_callback - ERROR - Error processing callback for node 69c33d1218a3717aa65d2fb6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56690 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48124 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:42:02,974 - lava_callback - INFO - Received callback for node 69c33d6418a3717aa65d2fc2 INFO: 172.20.0.1:42270 - "POST /node/69c33d6418a3717aa65d2fc2 HTTP/1.0" 202 Accepted 2026-03-27 01:42:02,982 - lava_callback - INFO - Starting async job submit for node 69c33d6418a3717aa65d2fc2 2026-03-27 01:42:02,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:42:02,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c33d6418a3717aa65d2fc2 HTTP/1.1" 200 None 2026-03-27 01:42:02,998 - lava_callback - DEBUG - Processing node 69c33d6418a3717aa65d2fc2: baseline-arm64-clabbe 2026-03-27 01:42:02,998 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c33d6418a3717aa65d2fc2 2026-03-27 01:42:02,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:42:03,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c33d6418a3717aa65d2fc2 HTTP/1.1" 200 None 2026-03-27 01:42:03,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:42:03,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-27 01:42:03,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:42:03,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c33d6418a3717aa65d2fc2 HTTP/1.1" 200 None 2026-03-27 01:42:03,064 - lava_callback - ERROR - Error processing callback for node 69c33d6418a3717aa65d2fc2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60296 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60306 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:42:17,078 - lava_callback - INFO - Received callback for node 69c5dc3b2cb62322ad968b74 INFO: 172.20.0.1:60318 - "POST /node/69c5dc3b2cb62322ad968b74 HTTP/1.0" 202 Accepted 2026-03-27 01:42:17,085 - lava_callback - INFO - Starting async job submit for node 69c5dc3b2cb62322ad968b74 2026-03-27 01:42:17,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:42:17,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5dc3b2cb62322ad968b74 HTTP/1.1" 200 None 2026-03-27 01:42:17,153 - lava_callback - DEBUG - Processing node 69c5dc3b2cb62322ad968b74: baseline-arm 2026-03-27 01:42:17,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 01:42:18,058 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:42:18,060 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c5dc3b2cb62322ad968b74/log.txt.gz 2026-03-27 01:42:18,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 01:42:18,736 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:42:18,738 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c5dc3b2cb62322ad968b74/lava-logs.yaml 2026-03-27 01:42:18,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 01:42:18,938 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:42:18,940 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c5dc3b2cb62322ad968b74/lava_callback.json.gz DEBUG: Setup failed for 69c5dc3b2cb62322ad968b74. Transitting job node result: incomplete -> fail 2026-03-27 01:42:19,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:42:19,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5dc3b2cb62322ad968b74 HTTP/1.1" 200 None 2026-03-27 01:42:19,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:42:19,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-27 01:42:19,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:42:19,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5dc3b2cb62322ad968b74 HTTP/1.1" 200 None 2026-03-27 01:42:19,227 - lava_callback - INFO - Completed processing callback for node 69c5dc3b2cb62322ad968b74 INFO: 172.20.0.1:48484 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:42:30,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:42:30,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:59674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57192 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58240 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57330 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:45:10,828 - lava_callback - INFO - Received callback for node 69c33e1f18a3717aa65d2fec INFO: 172.20.0.1:35360 - "POST /node/69c33e1f18a3717aa65d2fec HTTP/1.0" 202 Accepted 2026-03-27 01:45:10,836 - lava_callback - INFO - Starting async job submit for node 69c33e1f18a3717aa65d2fec 2026-03-27 01:45:10,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:45:10,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c33e1f18a3717aa65d2fec HTTP/1.1" 200 None 2026-03-27 01:45:10,900 - lava_callback - DEBUG - Processing node 69c33e1f18a3717aa65d2fec: baseline-arm64-baylibre 2026-03-27 01:45:10,900 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c33e1f18a3717aa65d2fec 2026-03-27 01:45:10,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:45:10,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c33e1f18a3717aa65d2fec HTTP/1.1" 200 None 2026-03-27 01:45:10,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:45:10,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-27 01:45:10,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:45:10,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c33e1f18a3717aa65d2fec HTTP/1.1" 200 None 2026-03-27 01:45:10,979 - lava_callback - ERROR - Error processing callback for node 69c33e1f18a3717aa65d2fec: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35384 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59488 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59496 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:46:18,527 - lava_callback - INFO - Received callback for node 69c5dcbc2cb62322ad968b89 INFO: 172.20.0.1:59498 - "POST /node/69c5dcbc2cb62322ad968b89 HTTP/1.0" 202 Accepted 2026-03-27 01:46:18,535 - lava_callback - INFO - Starting async job submit for node 69c5dcbc2cb62322ad968b89 2026-03-27 01:46:18,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:46:18,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5dcbc2cb62322ad968b89 HTTP/1.1" 200 None 2026-03-27 01:46:18,593 - lava_callback - DEBUG - Processing node 69c5dcbc2cb62322ad968b89: baseline-nfs-arm64 2026-03-27 01:46:19,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 01:46:20,164 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:46:20,165 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5dcbc2cb62322ad968b89/log.txt.gz 2026-03-27 01:46:20,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 01:46:21,104 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:46:21,106 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5dcbc2cb62322ad968b89/lava-logs.yaml 2026-03-27 01:46:21,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 01:46:21,326 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:46:21,328 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5dcbc2cb62322ad968b89/lava_callback.json.gz DEBUG: Setup failed for 69c5dcbc2cb62322ad968b89. Transitting job node result: incomplete -> fail 2026-03-27 01:46:21,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:46:21,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5dcbc2cb62322ad968b89 HTTP/1.1" 200 None 2026-03-27 01:46:21,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:46:21,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-27 01:46:21,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:46:21,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5dcbc2cb62322ad968b89 HTTP/1.1" 200 None 2026-03-27 01:46:21,588 - lava_callback - INFO - Completed processing callback for node 69c5dcbc2cb62322ad968b89 INFO: 172.20.0.1:58710 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:46:30,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:46:30,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:40982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46258 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50630 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:47:50,862 - lava_callback - INFO - Received callback for node 69c33ec018a3717aa65d3015 INFO: 172.20.0.1:47460 - "POST /node/69c33ec018a3717aa65d3015 HTTP/1.0" 202 Accepted 2026-03-27 01:47:50,868 - lava_callback - INFO - Starting async job submit for node 69c33ec018a3717aa65d3015 2026-03-27 01:47:50,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:47:50,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c33ec018a3717aa65d3015 HTTP/1.1" 200 None 2026-03-27 01:47:50,884 - lava_callback - DEBUG - Processing node 69c33ec018a3717aa65d3015: baseline-arm64-baylibre 2026-03-27 01:47:50,884 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c33ec018a3717aa65d3015 2026-03-27 01:47:50,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:47:50,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c33ec018a3717aa65d3015 HTTP/1.1" 200 None 2026-03-27 01:47:50,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:47:50,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-27 01:47:50,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:47:50,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c33ec018a3717aa65d3015 HTTP/1.1" 200 None 2026-03-27 01:47:50,959 - lava_callback - ERROR - Error processing callback for node 69c33ec018a3717aa65d3015: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46086 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46098 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:48:22,969 - lava_callback - INFO - Received callback for node 69c5e1ba2cb62322ad968cef INFO: 172.20.0.1:53242 - "POST /node/69c5e1ba2cb62322ad968cef HTTP/1.0" 202 Accepted 2026-03-27 01:48:22,978 - lava_callback - INFO - Starting async job submit for node 69c5e1ba2cb62322ad968cef 2026-03-27 01:48:23,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:48:23,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5e1ba2cb62322ad968cef HTTP/1.1" 200 None 2026-03-27 01:48:23,024 - lava_callback - DEBUG - Processing node 69c5e1ba2cb62322ad968cef: baseline-arm64 2026-03-27 01:48:23,177 - lava_callback - WARNING - Failed to upload log 2026-03-27 01:48:23,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 01:48:23,627 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:48:23,629 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5e1ba2cb62322ad968cef/lava-logs.yaml 2026-03-27 01:48:23,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 01:48:23,753 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:48:23,755 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5e1ba2cb62322ad968cef/lava_callback.json.gz 2026-03-27 01:48:23,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:48:23,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5e1ba2cb62322ad968cef HTTP/1.1" 200 None 2026-03-27 01:48:23,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:48:23,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-27 01:48:23,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:48:23,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5e1ba2cb62322ad968cef HTTP/1.1" 200 None 2026-03-27 01:48:23,911 - lava_callback - INFO - Completed processing callback for node 69c5e1ba2cb62322ad968cef INFO: 172.20.0.1:53254 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:48:30,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:48:30,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:44220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39498 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52990 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:49:51,077 - lava_callback - INFO - Received callback for node 69c33f3418a3717aa65d3027 INFO: 172.20.0.1:41436 - "POST /node/69c33f3418a3717aa65d3027 HTTP/1.0" 202 Accepted 2026-03-27 01:49:51,083 - lava_callback - INFO - Starting async job submit for node 69c33f3418a3717aa65d3027 2026-03-27 01:49:51,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:49:51,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c33f3418a3717aa65d3027 HTTP/1.1" 200 None 2026-03-27 01:49:51,127 - lava_callback - DEBUG - Processing node 69c33f3418a3717aa65d3027: baseline-arm64-baylibre 2026-03-27 01:49:51,127 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c33f3418a3717aa65d3027 2026-03-27 01:49:51,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:49:51,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c33f3418a3717aa65d3027 HTTP/1.1" 200 None 2026-03-27 01:49:51,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:49:51,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-27 01:49:51,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:49:51,210 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c33f3418a3717aa65d3027 HTTP/1.1" 200 None 2026-03-27 01:49:51,217 - lava_callback - ERROR - Error processing callback for node 69c33f3418a3717aa65d3027: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41452 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:49:58,226 - lava_callback - INFO - Received callback for node 69c5e1262cb62322ad968cbe INFO: 172.20.0.1:41466 - "POST /node/69c5e1262cb62322ad968cbe HTTP/1.0" 202 Accepted 2026-03-27 01:49:58,235 - lava_callback - INFO - Starting async job submit for node 69c5e1262cb62322ad968cbe 2026-03-27 01:49:58,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:49:58,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5e1262cb62322ad968cbe HTTP/1.1" 200 None 2026-03-27 01:49:58,300 - lava_callback - DEBUG - Processing node 69c5e1262cb62322ad968cbe: baseline-arm64-foundriesio 2026-03-27 01:49:59,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 01:49:59,612 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:49:59,613 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c5e1262cb62322ad968cbe/log.txt.gz 2026-03-27 01:50:00,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 01:50:00,566 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:50:00,568 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c5e1262cb62322ad968cbe/lava-logs.yaml 2026-03-27 01:50:00,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 01:50:00,780 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:50:00,782 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c5e1262cb62322ad968cbe/lava_callback.json.gz DEBUG: 69c5e1262cb62322ad968cbe Transitting job node result: pass -> incomplete 2026-03-27 01:50:00,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:50:00,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5e1262cb62322ad968cbe HTTP/1.1" 200 None 2026-03-27 01:50:00,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:50:00,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-27 01:50:00,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:50:00,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5e1262cb62322ad968cbe HTTP/1.1" 200 None 2026-03-27 01:50:00,898 - lava_callback - INFO - Completed processing callback for node 69c5e1262cb62322ad968cbe 2026-03-27 01:50:00,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:50:00,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:47540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57872 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45302 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:50:43,079 - lava_callback - INFO - Received callback for node 69c5e2692cb62322ad968d41 INFO: 172.20.0.1:45932 - "POST /node/69c5e2692cb62322ad968d41 HTTP/1.0" 202 Accepted 2026-03-27 01:50:43,085 - lava_callback - INFO - Starting async job submit for node 69c5e2692cb62322ad968d41 2026-03-27 01:50:43,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:50:43,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5e2692cb62322ad968d41 HTTP/1.1" 200 None 2026-03-27 01:50:43,102 - lava_callback - DEBUG - Processing node 69c5e2692cb62322ad968d41: baseline-nfs-arm64 2026-03-27 01:50:43,103 - lava_callback - WARNING - Failed to upload log 2026-03-27 01:50:43,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 01:50:43,239 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:50:43,240 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5e2692cb62322ad968d41/lava-logs.yaml 2026-03-27 01:50:43,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 01:50:43,358 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:50:43,360 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5e2692cb62322ad968d41/lava_callback.json.gz 2026-03-27 01:50:43,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:50:43,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5e2692cb62322ad968d41 HTTP/1.1" 200 None 2026-03-27 01:50:43,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:50:43,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-27 01:50:43,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:50:43,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5e2692cb62322ad968d41 HTTP/1.1" 200 None 2026-03-27 01:50:43,444 - lava_callback - INFO - Completed processing callback for node 69c5e2692cb62322ad968d41 INFO: 172.20.0.1:45940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52382 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:51:00,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:51:00,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:54238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53838 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49088 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:51:46,753 - lava_callback - INFO - Received callback for node 69c5d8342cb62322ad968a8b INFO: 172.20.0.1:49098 - "POST /node/69c5d8342cb62322ad968a8b HTTP/1.0" 202 Accepted 2026-03-27 01:51:46,801 - lava_callback - INFO - Starting async job submit for node 69c5d8342cb62322ad968a8b 2026-03-27 01:51:46,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:51:46,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d8342cb62322ad968a8b HTTP/1.1" 200 None 2026-03-27 01:51:46,842 - lava_callback - DEBUG - Processing node 69c5d8342cb62322ad968a8b: baseline-arm-pengutronix 2026-03-27 01:51:46,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 01:51:47,099 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:51:47,100 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c5d8342cb62322ad968a8b/log.txt.gz 2026-03-27 01:51:47,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 01:51:47,355 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:51:47,357 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c5d8342cb62322ad968a8b/lava-logs.yaml 2026-03-27 01:51:47,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 01:51:47,513 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:51:47,515 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c5d8342cb62322ad968a8b/lava_callback.json.gz DEBUG: Setup failed for 69c5d8342cb62322ad968a8b. Transitting job node result: incomplete -> fail 2026-03-27 01:51:47,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:51:47,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d8342cb62322ad968a8b HTTP/1.1" 200 None 2026-03-27 01:51:47,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:51:47,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-27 01:51:47,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:51:47,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d8342cb62322ad968a8b HTTP/1.1" 200 None 2026-03-27 01:51:47,696 - lava_callback - INFO - Completed processing callback for node 69c5d8342cb62322ad968a8b INFO: 172.20.0.1:35292 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:52:00,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:52:00,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:56326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37510 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35874 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:52:31,253 - lava_callback - INFO - Received callback for node 69c33fd318a3717aa65d303a INFO: 172.20.0.1:38644 - "POST /node/69c33fd318a3717aa65d303a HTTP/1.0" 202 Accepted 2026-03-27 01:52:31,259 - lava_callback - INFO - Starting async job submit for node 69c33fd318a3717aa65d303a 2026-03-27 01:52:31,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:52:31,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c33fd318a3717aa65d303a HTTP/1.1" 200 None 2026-03-27 01:52:31,276 - lava_callback - DEBUG - Processing node 69c33fd318a3717aa65d303a: baseline-arm64-baylibre 2026-03-27 01:52:31,276 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c33fd318a3717aa65d303a 2026-03-27 01:52:31,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:52:31,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c33fd318a3717aa65d303a HTTP/1.1" 200 None 2026-03-27 01:52:31,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:52:31,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-27 01:52:31,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:52:31,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c33fd318a3717aa65d303a HTTP/1.1" 200 None 2026-03-27 01:52:31,337 - lava_callback - ERROR - Error processing callback for node 69c33fd318a3717aa65d303a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57352 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:53:04,908 - lava_callback - INFO - Received callback for node 69c5e2b52cb62322ad968d59 INFO: 172.20.0.1:32812 - "POST /node/69c5e2b52cb62322ad968d59 HTTP/1.0" 202 Accepted 2026-03-27 01:53:04,915 - lava_callback - INFO - Starting async job submit for node 69c5e2b52cb62322ad968d59 2026-03-27 01:53:04,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:53:04,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5e2b52cb62322ad968d59 HTTP/1.1" 200 None 2026-03-27 01:53:04,960 - lava_callback - DEBUG - Processing node 69c5e2b52cb62322ad968d59: baseline-arm64 2026-03-27 01:53:05,162 - lava_callback - WARNING - Failed to upload log 2026-03-27 01:53:05,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 01:53:05,521 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:53:05,523 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5e2b52cb62322ad968d59/lava-logs.yaml 2026-03-27 01:53:05,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 01:53:05,668 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:53:05,669 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5e2b52cb62322ad968d59/lava_callback.json.gz 2026-03-27 01:53:05,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:53:05,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5e2b52cb62322ad968d59 HTTP/1.1" 200 None 2026-03-27 01:53:05,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:53:05,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-27 01:53:05,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:53:05,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5e2b52cb62322ad968d59 HTTP/1.1" 200 None 2026-03-27 01:53:05,844 - lava_callback - INFO - Completed processing callback for node 69c5e2b52cb62322ad968d59 INFO: 172.20.0.1:32820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52938 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56292 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:53:30,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:53:31,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:47618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44198 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.20.0.1:51282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51284 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59054 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:55:11,302 - lava_callback - INFO - Received callback for node 69c3407418a3717aa65d3048 INFO: 172.20.0.1:32952 - "POST /node/69c3407418a3717aa65d3048 HTTP/1.0" 202 Accepted 2026-03-27 01:55:11,307 - lava_callback - INFO - Starting async job submit for node 69c3407418a3717aa65d3048 2026-03-27 01:55:11,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:55:11,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3407418a3717aa65d3048 HTTP/1.1" 200 None 2026-03-27 01:55:11,323 - lava_callback - DEBUG - Processing node 69c3407418a3717aa65d3048: baseline-arm64-baylibre 2026-03-27 01:55:11,324 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3407418a3717aa65d3048 2026-03-27 01:55:11,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:55:11,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3407418a3717aa65d3048 HTTP/1.1" 200 None 2026-03-27 01:55:11,339 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:55:11,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-27 01:55:11,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:55:11,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3407418a3717aa65d3048 HTTP/1.1" 200 None 2026-03-27 01:55:11,388 - lava_callback - ERROR - Error processing callback for node 69c3407418a3717aa65d3048: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:32954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32960 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43740 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33222 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:56:42,764 - lava_callback - INFO - Received callback for node 69c5e2132cb62322ad968d15 INFO: 172.20.0.1:34356 - "POST /node/69c5e2132cb62322ad968d15 HTTP/1.0" 202 Accepted 2026-03-27 01:56:42,772 - lava_callback - INFO - Starting async job submit for node 69c5e2132cb62322ad968d15 2026-03-27 01:56:42,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:56:42,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5e2132cb62322ad968d15 HTTP/1.1" 200 None 2026-03-27 01:56:42,829 - lava_callback - DEBUG - Processing node 69c5e2132cb62322ad968d15: baseline-arm64 2026-03-27 01:56:43,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 01:56:43,526 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:56:43,528 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5e2132cb62322ad968d15/log.txt.gz 2026-03-27 01:56:43,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 01:56:43,993 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:56:43,995 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5e2132cb62322ad968d15/lava-logs.yaml 2026-03-27 01:56:44,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 01:56:44,171 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 01:56:44,173 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5e2132cb62322ad968d15/lava_callback.json.gz DEBUG: Setup failed for 69c5e2132cb62322ad968d15. Transitting job node result: incomplete -> fail 2026-03-27 01:56:44,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:56:44,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5e2132cb62322ad968d15 HTTP/1.1" 200 None 2026-03-27 01:56:44,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:56:44,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-27 01:56:44,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:56:44,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5e2132cb62322ad968d15 HTTP/1.1" 200 None 2026-03-27 01:56:44,417 - lava_callback - INFO - Completed processing callback for node 69c5e2132cb62322ad968d15 INFO: 172.20.0.1:34368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45346 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:57:01,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:57:01,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:41458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40154 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47908 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:57:51,625 - lava_callback - INFO - Received callback for node 69c3411618a3717aa65d3064 INFO: 172.20.0.1:49292 - "POST /node/69c3411618a3717aa65d3064 HTTP/1.0" 202 Accepted 2026-03-27 01:57:51,631 - lava_callback - INFO - Starting async job submit for node 69c3411618a3717aa65d3064 2026-03-27 01:57:51,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:57:51,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3411618a3717aa65d3064 HTTP/1.1" 200 None 2026-03-27 01:57:51,651 - lava_callback - DEBUG - Processing node 69c3411618a3717aa65d3064: baseline-arm64-baylibre 2026-03-27 01:57:51,651 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3411618a3717aa65d3064 2026-03-27 01:57:51,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:57:51,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3411618a3717aa65d3064 HTTP/1.1" 200 None 2026-03-27 01:57:51,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:57:51,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-27 01:57:51,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:57:51,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3411618a3717aa65d3064 HTTP/1.1" 200 None 2026-03-27 01:57:51,713 - lava_callback - ERROR - Error processing callback for node 69c3411618a3717aa65d3064: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40672 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58638 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58642 - "GET / HTTP/1.0" 200 OK 2026-03-27 01:59:23,903 - lava_callback - INFO - Received callback for node 69c3416c18a3717aa65d3074 INFO: 172.20.0.1:40826 - "POST /node/69c3416c18a3717aa65d3074 HTTP/1.0" 202 Accepted 2026-03-27 01:59:23,909 - lava_callback - INFO - Starting async job submit for node 69c3416c18a3717aa65d3074 2026-03-27 01:59:23,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:59:23,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3416c18a3717aa65d3074 HTTP/1.1" 200 None 2026-03-27 01:59:23,925 - lava_callback - DEBUG - Processing node 69c3416c18a3717aa65d3074: baseline-arm64-clabbe 2026-03-27 01:59:23,925 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3416c18a3717aa65d3074 2026-03-27 01:59:23,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:59:23,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3416c18a3717aa65d3074 HTTP/1.1" 200 None 2026-03-27 01:59:23,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:59:23,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-27 01:59:23,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 01:59:23,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3416c18a3717aa65d3074 HTTP/1.1" 200 None 2026-03-27 01:59:23,995 - lava_callback - ERROR - Error processing callback for node 69c3416c18a3717aa65d3074: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44600 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60384 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52716 - "GET / HTTP/1.0" 200 OK 2026-03-27 02:01:27,106 - lava_callback - INFO - Received callback for node 69c51634f63ce05331e2b650 INFO: 172.20.0.1:52726 - "POST /node/69c51634f63ce05331e2b650 HTTP/1.0" 202 Accepted 2026-03-27 02:01:27,113 - lava_callback - INFO - Starting async job submit for node 69c51634f63ce05331e2b650 2026-03-27 02:01:27,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:01:27,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c51634f63ce05331e2b650 HTTP/1.1" 200 None 2026-03-27 02:01:27,161 - lava_callback - DEBUG - Processing node 69c51634f63ce05331e2b650: baseline-arm64 2026-03-27 02:01:27,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 02:01:27,843 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 02:01:27,844 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c51634f63ce05331e2b650/log.txt.gz 2026-03-27 02:01:28,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 02:01:28,340 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 02:01:28,342 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c51634f63ce05331e2b650/lava-logs.yaml 2026-03-27 02:01:28,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 02:01:28,536 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 02:01:28,538 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c51634f63ce05331e2b650/lava_callback.json.gz 2026-03-27 02:01:28,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:01:28,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c51634f63ce05331e2b650 HTTP/1.1" 200 None 2026-03-27 02:01:28,586 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:01:28,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-27 02:01:28,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:01:28,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c51634f63ce05331e2b650 HTTP/1.1" 200 None 2026-03-27 02:01:28,679 - lava_callback - INFO - Completed processing callback for node 69c51634f63ce05331e2b650 2026-03-27 02:01:31,030 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:01:31,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:53758 - "GET / HTTP/1.0" 200 OK 2026-03-27 02:01:40,483 - lava_callback - INFO - Received callback for node 69c5e49a2cb62322ad968dea INFO: 172.20.0.1:55530 - "POST /node/69c5e49a2cb62322ad968dea HTTP/1.0" 202 Accepted 2026-03-27 02:01:40,490 - lava_callback - INFO - Starting async job submit for node 69c5e49a2cb62322ad968dea 2026-03-27 02:01:40,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:01:40,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5e49a2cb62322ad968dea HTTP/1.1" 200 None 2026-03-27 02:01:40,556 - lava_callback - DEBUG - Processing node 69c5e49a2cb62322ad968dea: baseline-arm64-foundriesio 2026-03-27 02:01:40,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 02:01:41,145 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 02:01:41,147 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c5e49a2cb62322ad968dea/log.txt.gz 2026-03-27 02:01:41,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 02:01:41,755 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 02:01:41,757 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c5e49a2cb62322ad968dea/lava-logs.yaml 2026-03-27 02:01:41,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 02:01:41,907 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 02:01:41,909 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c5e49a2cb62322ad968dea/lava_callback.json.gz 2026-03-27 02:01:41,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:01:41,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5e49a2cb62322ad968dea HTTP/1.1" 200 None 2026-03-27 02:01:41,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:01:41,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-27 02:01:41,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:01:42,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5e49a2cb62322ad968dea HTTP/1.1" 200 None 2026-03-27 02:01:42,007 - lava_callback - INFO - Completed processing callback for node 69c5e49a2cb62322ad968dea INFO: 172.20.0.1:55534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40030 - "GET / HTTP/1.0" 200 OK 2026-03-27 02:02:01,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:02:01,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-27 02:02:03,752 - lava_callback - INFO - Received callback for node 69c3421318a3717aa65d3088 INFO: 172.20.0.1:54008 - "POST /node/69c3421318a3717aa65d3088 HTTP/1.0" 202 Accepted 2026-03-27 02:02:03,758 - lava_callback - INFO - Starting async job submit for node 69c3421318a3717aa65d3088 2026-03-27 02:02:03,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:02:03,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3421318a3717aa65d3088 HTTP/1.1" 200 None 2026-03-27 02:02:03,773 - lava_callback - DEBUG - Processing node 69c3421318a3717aa65d3088: baseline-arm64-clabbe 2026-03-27 02:02:03,773 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3421318a3717aa65d3088 2026-03-27 02:02:03,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:02:03,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3421318a3717aa65d3088 HTTP/1.1" 200 None 2026-03-27 02:02:03,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:02:03,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-27 02:02:03,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:02:03,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3421318a3717aa65d3088 HTTP/1.1" 200 None 2026-03-27 02:02:03,824 - lava_callback - ERROR - Error processing callback for node 69c3421318a3717aa65d3088: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44348 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39154 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39160 - "GET / HTTP/1.0" 200 OK 2026-03-27 02:03:23,803 - lava_callback - INFO - Received callback for node 69c3426618a3717aa65d3095 INFO: 172.20.0.1:42326 - "POST /node/69c3426618a3717aa65d3095 HTTP/1.0" 202 Accepted 2026-03-27 02:03:23,809 - lava_callback - INFO - Starting async job submit for node 69c3426618a3717aa65d3095 2026-03-27 02:03:23,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:03:23,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3426618a3717aa65d3095 HTTP/1.1" 200 None 2026-03-27 02:03:23,824 - lava_callback - DEBUG - Processing node 69c3426618a3717aa65d3095: baseline-arm64-clabbe 2026-03-27 02:03:23,824 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3426618a3717aa65d3095 2026-03-27 02:03:23,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:03:23,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3426618a3717aa65d3095 HTTP/1.1" 200 None 2026-03-27 02:03:23,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:03:23,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-27 02:03:23,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:03:23,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3426618a3717aa65d3095 HTTP/1.1" 200 None 2026-03-27 02:03:23,880 - lava_callback - ERROR - Error processing callback for node 69c3426618a3717aa65d3095: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34088 - "GET / HTTP/1.0" 200 OK 2026-03-27 02:03:56,934 - lava_callback - INFO - Received callback for node 69c5e1642cb62322ad968cc5 INFO: 172.20.0.1:34098 - "POST /node/69c5e1642cb62322ad968cc5 HTTP/1.0" 202 Accepted 2026-03-27 02:03:56,941 - lava_callback - INFO - Starting async job submit for node 69c5e1642cb62322ad968cc5 2026-03-27 02:03:56,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:03:57,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5e1642cb62322ad968cc5 HTTP/1.1" 200 None 2026-03-27 02:03:57,014 - lava_callback - DEBUG - Processing node 69c5e1642cb62322ad968cc5: ltp-smoke 2026-03-27 02:03:57,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 02:03:57,792 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 02:03:57,794 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c5e1642cb62322ad968cc5/log.txt.gz 2026-03-27 02:03:58,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 02:03:58,319 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 02:03:58,321 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c5e1642cb62322ad968cc5/lava-logs.yaml 2026-03-27 02:03:58,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 02:03:58,538 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 02:03:58,540 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c5e1642cb62322ad968cc5/lava_callback.json.gz 2026-03-27 02:03:58,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:03:58,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5e1642cb62322ad968cc5 HTTP/1.1" 200 None 2026-03-27 02:03:58,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:03:58,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc312cb62322ad965dad HTTP/1.1" 200 None 2026-03-27 02:03:58,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:03:58,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5e1642cb62322ad968cc5 HTTP/1.1" 200 None 2026-03-27 02:03:58,850 - lava_callback - INFO - Completed processing callback for node 69c5e1642cb62322ad968cc5 2026-03-27 02:04:01,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:04:01,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:41134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46298 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45122 - "GET / HTTP/1.0" 200 OK 2026-03-27 02:04:43,837 - lava_callback - INFO - Received callback for node 69c342b618a3717aa65d30a6 INFO: 172.20.0.1:49732 - "POST /node/69c342b618a3717aa65d30a6 HTTP/1.0" 202 Accepted 2026-03-27 02:04:43,842 - lava_callback - INFO - Starting async job submit for node 69c342b618a3717aa65d30a6 2026-03-27 02:04:43,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:04:43,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c342b618a3717aa65d30a6 HTTP/1.1" 200 None 2026-03-27 02:04:43,857 - lava_callback - DEBUG - Processing node 69c342b618a3717aa65d30a6: baseline-arm64-clabbe 2026-03-27 02:04:43,857 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c342b618a3717aa65d30a6 2026-03-27 02:04:43,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:04:43,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c342b618a3717aa65d30a6 HTTP/1.1" 200 None 2026-03-27 02:04:43,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:04:43,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-27 02:04:43,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:04:43,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c342b618a3717aa65d30a6 HTTP/1.1" 200 None 2026-03-27 02:04:43,917 - lava_callback - ERROR - Error processing callback for node 69c342b618a3717aa65d30a6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55802 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55814 - "GET / HTTP/1.0" 200 OK 2026-03-27 02:05:21,596 - lava_callback - INFO - Received callback for node 69c54c33f63ce05331e2be13 INFO: 172.20.0.1:40822 - "POST /node/69c54c33f63ce05331e2be13 HTTP/1.0" 202 Accepted 2026-03-27 02:05:21,602 - lava_callback - INFO - Starting async job submit for node 69c54c33f63ce05331e2be13 2026-03-27 02:05:21,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:05:21,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c54c33f63ce05331e2be13 HTTP/1.1" 200 None 2026-03-27 02:05:21,650 - lava_callback - DEBUG - Processing node 69c54c33f63ce05331e2be13: baseline-arm64 2026-03-27 02:05:22,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 02:05:22,298 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 02:05:22,299 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c54c33f63ce05331e2be13/log.txt.gz 2026-03-27 02:05:22,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 02:05:22,790 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 02:05:22,793 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c54c33f63ce05331e2be13/lava-logs.yaml 2026-03-27 02:05:22,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 02:05:22,986 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 02:05:22,988 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c54c33f63ce05331e2be13/lava_callback.json.gz 2026-03-27 02:05:23,015 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:05:23,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c54c33f63ce05331e2be13 HTTP/1.1" 200 None 2026-03-27 02:05:23,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:05:23,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-27 02:05:23,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:05:23,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c54c33f63ce05331e2be13 HTTP/1.1" 200 None 2026-03-27 02:05:23,128 - lava_callback - INFO - Completed processing callback for node 69c54c33f63ce05331e2be13 INFO: 172.20.0.1:40838 - "GET / HTTP/1.0" 200 OK 2026-03-27 02:05:31,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:05:31,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:37968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53402 - "GET / HTTP/1.0" 200 OK 2026-03-27 02:06:03,905 - lava_callback - INFO - Received callback for node 69c3430818a3717aa65d30b6 INFO: 172.20.0.1:43654 - "POST /node/69c3430818a3717aa65d30b6 HTTP/1.0" 202 Accepted 2026-03-27 02:06:03,911 - lava_callback - INFO - Starting async job submit for node 69c3430818a3717aa65d30b6 2026-03-27 02:06:03,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:06:03,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3430818a3717aa65d30b6 HTTP/1.1" 200 None 2026-03-27 02:06:03,928 - lava_callback - DEBUG - Processing node 69c3430818a3717aa65d30b6: baseline-arm64-clabbe 2026-03-27 02:06:03,928 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3430818a3717aa65d30b6 2026-03-27 02:06:03,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:06:03,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3430818a3717aa65d30b6 HTTP/1.1" 200 None 2026-03-27 02:06:03,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:06:03,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-27 02:06:03,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:06:03,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3430818a3717aa65d30b6 HTTP/1.1" 200 None 2026-03-27 02:06:03,990 - lava_callback - ERROR - Error processing callback for node 69c3430818a3717aa65d30b6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33080 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41834 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34794 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46846 - "GET / HTTP/1.0" 200 OK 2026-03-27 02:09:04,090 - lava_callback - INFO - Received callback for node 69c343aa18a3717aa65d30cf INFO: 172.20.0.1:46822 - "POST /node/69c343aa18a3717aa65d30cf HTTP/1.0" 202 Accepted 2026-03-27 02:09:04,096 - lava_callback - INFO - Starting async job submit for node 69c343aa18a3717aa65d30cf 2026-03-27 02:09:04,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:09:04,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c343aa18a3717aa65d30cf HTTP/1.1" 200 None 2026-03-27 02:09:04,112 - lava_callback - DEBUG - Processing node 69c343aa18a3717aa65d30cf: baseline-arm64-clabbe 2026-03-27 02:09:04,113 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c343aa18a3717aa65d30cf 2026-03-27 02:09:04,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:09:04,129 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c343aa18a3717aa65d30cf HTTP/1.1" 200 None 2026-03-27 02:09:04,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:09:04,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-27 02:09:04,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:09:04,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c343aa18a3717aa65d30cf HTTP/1.1" 200 None 2026-03-27 02:09:04,179 - lava_callback - ERROR - Error processing callback for node 69c343aa18a3717aa65d30cf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37536 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48942 - "GET / HTTP/1.0" 200 OK 2026-03-27 02:09:56,097 - lava_callback - INFO - Received callback for node 69c5e3082cb62322ad968d71 INFO: 172.20.0.1:56082 - "POST /node/69c5e3082cb62322ad968d71 HTTP/1.0" 202 Accepted 2026-03-27 02:09:56,104 - lava_callback - INFO - Starting async job submit for node 69c5e3082cb62322ad968d71 2026-03-27 02:09:56,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:09:56,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5e3082cb62322ad968d71 HTTP/1.1" 200 None 2026-03-27 02:09:56,171 - lava_callback - DEBUG - Processing node 69c5e3082cb62322ad968d71: baseline-arm INFO: 172.20.0.1:56086 - "GET / HTTP/1.0" 200 OK 2026-03-27 02:09:56,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 02:09:56,910 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 02:09:56,911 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c5e3082cb62322ad968d71/log.txt.gz 2026-03-27 02:09:57,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 02:09:57,534 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 02:09:57,536 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c5e3082cb62322ad968d71/lava-logs.yaml 2026-03-27 02:09:57,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 02:09:57,703 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 02:09:57,704 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c5e3082cb62322ad968d71/lava_callback.json.gz DEBUG: Setup failed for 69c5e3082cb62322ad968d71. Transitting job node result: incomplete -> fail 2026-03-27 02:09:57,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:09:57,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5e3082cb62322ad968d71 HTTP/1.1" 200 None 2026-03-27 02:09:57,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:09:57,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-27 02:09:57,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:09:57,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5e3082cb62322ad968d71 HTTP/1.1" 200 None 2026-03-27 02:09:57,993 - lava_callback - INFO - Completed processing callback for node 69c5e3082cb62322ad968d71 2026-03-27 02:10:01,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:10:01,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:33248 - "GET / HTTP/1.0" 200 OK 2026-03-27 02:10:12,121 - lava_callback - INFO - Received callback for node 69c343f218a3717aa65d30db INFO: 172.20.0.1:46690 - "POST /node/69c343f218a3717aa65d30db HTTP/1.0" 202 Accepted 2026-03-27 02:10:12,126 - lava_callback - INFO - Starting async job submit for node 69c343f218a3717aa65d30db 2026-03-27 02:10:12,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:10:12,146 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c343f218a3717aa65d30db HTTP/1.1" 200 None 2026-03-27 02:10:12,147 - lava_callback - DEBUG - Processing node 69c343f218a3717aa65d30db: baseline-arm64-baylibre 2026-03-27 02:10:12,147 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c343f218a3717aa65d30db 2026-03-27 02:10:12,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:10:12,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c343f218a3717aa65d30db HTTP/1.1" 200 None 2026-03-27 02:10:12,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:10:12,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086b1ce0c9bc43bccb63a HTTP/1.1" 200 None 2026-03-27 02:10:12,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:10:12,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c343f218a3717aa65d30db HTTP/1.1" 200 None 2026-03-27 02:10:12,199 - lava_callback - ERROR - Error processing callback for node 69c343f218a3717aa65d30db: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46696 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58536 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37068 - "GET / HTTP/1.0" 200 OK 2026-03-27 02:11:32,602 - lava_callback - INFO - Received callback for node 69c3444218a3717aa65d30e0 INFO: 172.20.0.1:44690 - "POST /node/69c3444218a3717aa65d30e0 HTTP/1.0" 202 Accepted 2026-03-27 02:11:32,608 - lava_callback - INFO - Starting async job submit for node 69c3444218a3717aa65d30e0 2026-03-27 02:11:32,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:11:32,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3444218a3717aa65d30e0 HTTP/1.1" 200 None 2026-03-27 02:11:32,623 - lava_callback - DEBUG - Processing node 69c3444218a3717aa65d30e0: baseline-arm64-baylibre 2026-03-27 02:11:32,624 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3444218a3717aa65d30e0 2026-03-27 02:11:32,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:11:32,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3444218a3717aa65d30e0 HTTP/1.1" 200 None 2026-03-27 02:11:32,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:11:32,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086b1ce0c9bc43bccb63a HTTP/1.1" 200 None 2026-03-27 02:11:32,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:11:32,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3444218a3717aa65d30e0 HTTP/1.1" 200 None 2026-03-27 02:11:32,685 - lava_callback - ERROR - Error processing callback for node 69c3444218a3717aa65d30e0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41138 - "GET / HTTP/1.0" 200 OK 2026-03-27 02:11:56,581 - lava_callback - INFO - Received callback for node 69c5e5ad2cb62322ad968e36 INFO: 172.20.0.1:41144 - "POST /node/69c5e5ad2cb62322ad968e36 HTTP/1.0" 202 Accepted 2026-03-27 02:11:56,589 - lava_callback - INFO - Starting async job submit for node 69c5e5ad2cb62322ad968e36 2026-03-27 02:11:56,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:11:56,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5e5ad2cb62322ad968e36 HTTP/1.1" 200 None 2026-03-27 02:11:56,642 - lava_callback - DEBUG - Processing node 69c5e5ad2cb62322ad968e36: baseline-arm64 2026-03-27 02:11:57,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 02:11:57,160 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 02:11:57,161 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5e5ad2cb62322ad968e36/log.txt.gz 2026-03-27 02:11:57,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 02:11:57,620 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 02:11:57,622 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5e5ad2cb62322ad968e36/lava-logs.yaml 2026-03-27 02:11:57,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 02:11:57,803 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 02:11:57,805 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5e5ad2cb62322ad968e36/lava_callback.json.gz DEBUG: Setup failed for 69c5e5ad2cb62322ad968e36. Transitting job node result: incomplete -> fail 2026-03-27 02:11:57,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:11:57,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5e5ad2cb62322ad968e36 HTTP/1.1" 200 None 2026-03-27 02:11:57,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:11:57,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-27 02:11:57,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:11:57,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5e5ad2cb62322ad968e36 HTTP/1.1" 200 None 2026-03-27 02:11:58,034 - lava_callback - INFO - Completed processing callback for node 69c5e5ad2cb62322ad968e36 2026-03-27 02:12:01,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:12:01,157 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:46830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45102 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55606 - "GET / HTTP/1.0" 200 OK 2026-03-27 02:12:52,303 - lava_callback - INFO - Received callback for node 69c3449518a3717aa65d30eb INFO: 172.20.0.1:44502 - "POST /node/69c3449518a3717aa65d30eb HTTP/1.0" 202 Accepted 2026-03-27 02:12:52,309 - lava_callback - INFO - Starting async job submit for node 69c3449518a3717aa65d30eb 2026-03-27 02:12:52,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:12:52,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3449518a3717aa65d30eb HTTP/1.1" 200 None 2026-03-27 02:12:52,325 - lava_callback - DEBUG - Processing node 69c3449518a3717aa65d30eb: baseline-arm64-baylibre 2026-03-27 02:12:52,325 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3449518a3717aa65d30eb 2026-03-27 02:12:52,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:12:52,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3449518a3717aa65d30eb HTTP/1.1" 200 None 2026-03-27 02:12:52,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:12:52,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086b1ce0c9bc43bccb63a HTTP/1.1" 200 None 2026-03-27 02:12:52,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:12:52,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3449518a3717aa65d30eb HTTP/1.1" 200 None 2026-03-27 02:12:52,414 - lava_callback - ERROR - Error processing callback for node 69c3449518a3717aa65d30eb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46310 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46316 - "GET / HTTP/1.0" 200 OK 2026-03-27 02:13:19,572 - lava_callback - INFO - Received callback for node 69c54c85f63ce05331e2be21 INFO: 172.20.0.1:46330 - "POST /node/69c54c85f63ce05331e2be21 HTTP/1.0" 202 Accepted 2026-03-27 02:13:19,579 - lava_callback - INFO - Starting async job submit for node 69c54c85f63ce05331e2be21 2026-03-27 02:13:19,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:13:19,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c54c85f63ce05331e2be21 HTTP/1.1" 200 None 2026-03-27 02:13:19,621 - lava_callback - DEBUG - Processing node 69c54c85f63ce05331e2be21: baseline-nfs-arm64 2026-03-27 02:13:20,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 02:13:20,295 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 02:13:20,297 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c54c85f63ce05331e2be21/log.txt.gz 2026-03-27 02:13:20,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 02:13:20,778 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 02:13:20,780 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c54c85f63ce05331e2be21/lava-logs.yaml 2026-03-27 02:13:20,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 02:13:21,124 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 02:13:21,125 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c54c85f63ce05331e2be21/lava_callback.json.gz 2026-03-27 02:13:21,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:13:21,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c54c85f63ce05331e2be21 HTTP/1.1" 200 None 2026-03-27 02:13:21,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:13:21,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-27 02:13:21,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:13:21,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c54c85f63ce05331e2be21 HTTP/1.1" 200 None 2026-03-27 02:13:21,299 - lava_callback - INFO - Completed processing callback for node 69c54c85f63ce05331e2be21 INFO: 172.20.0.1:41682 - "GET / HTTP/1.0" 200 OK 2026-03-27 02:13:31,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:13:31,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:38516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36814 - "GET / HTTP/1.0" 200 OK 2026-03-27 02:14:12,352 - lava_callback - INFO - Received callback for node 69c344e618a3717aa65d30f6 INFO: 172.20.0.1:44806 - "POST /node/69c344e618a3717aa65d30f6 HTTP/1.0" 202 Accepted 2026-03-27 02:14:12,358 - lava_callback - INFO - Starting async job submit for node 69c344e618a3717aa65d30f6 2026-03-27 02:14:12,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:14:12,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c344e618a3717aa65d30f6 HTTP/1.1" 200 None 2026-03-27 02:14:12,374 - lava_callback - DEBUG - Processing node 69c344e618a3717aa65d30f6: baseline-arm64-baylibre 2026-03-27 02:14:12,374 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c344e618a3717aa65d30f6 2026-03-27 02:14:12,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:14:12,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c344e618a3717aa65d30f6 HTTP/1.1" 200 None 2026-03-27 02:14:12,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:14:12,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086b1ce0c9bc43bccb63a HTTP/1.1" 200 None 2026-03-27 02:14:12,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:14:12,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c344e618a3717aa65d30f6 HTTP/1.1" 200 None 2026-03-27 02:14:12,433 - lava_callback - ERROR - Error processing callback for node 69c344e618a3717aa65d30f6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44810 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59440 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59454 - "GET / HTTP/1.0" 200 OK 2026-03-27 02:15:19,180 - lava_callback - INFO - Received callback for node 69c5e3882cb62322ad968d84 INFO: 172.20.0.1:59468 - "POST /node/69c5e3882cb62322ad968d84 HTTP/1.0" 202 Accepted 2026-03-27 02:15:19,188 - lava_callback - INFO - Starting async job submit for node 69c5e3882cb62322ad968d84 2026-03-27 02:15:19,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:15:19,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5e3882cb62322ad968d84 HTTP/1.1" 200 None 2026-03-27 02:15:19,260 - lava_callback - DEBUG - Processing node 69c5e3882cb62322ad968d84: baseline-nfs-arm64 2026-03-27 02:15:20,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 02:15:21,236 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 02:15:21,238 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5e3882cb62322ad968d84/log.txt.gz 2026-03-27 02:15:21,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 02:15:22,268 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 02:15:22,271 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5e3882cb62322ad968d84/lava-logs.yaml 2026-03-27 02:15:22,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 02:15:22,504 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 02:15:22,506 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5e3882cb62322ad968d84/lava_callback.json.gz DEBUG: Setup failed for 69c5e3882cb62322ad968d84. Transitting job node result: incomplete -> fail 2026-03-27 02:15:22,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:15:22,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5e3882cb62322ad968d84 HTTP/1.1" 200 None 2026-03-27 02:15:22,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:15:22,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-27 02:15:22,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:15:22,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5e3882cb62322ad968d84 HTTP/1.1" 200 None 2026-03-27 02:15:22,778 - lava_callback - INFO - Completed processing callback for node 69c5e3882cb62322ad968d84 INFO: 172.20.0.1:58710 - "GET / HTTP/1.0" 200 OK 2026-03-27 02:15:31,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:15:31,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-27 02:15:32,399 - lava_callback - INFO - Received callback for node 69c3453818a3717aa65d3103 INFO: 172.20.0.1:60178 - "POST /node/69c3453818a3717aa65d3103 HTTP/1.0" 202 Accepted 2026-03-27 02:15:32,405 - lava_callback - INFO - Starting async job submit for node 69c3453818a3717aa65d3103 2026-03-27 02:15:32,405 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:15:32,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3453818a3717aa65d3103 HTTP/1.1" 200 None 2026-03-27 02:15:32,424 - lava_callback - DEBUG - Processing node 69c3453818a3717aa65d3103: baseline-arm64-baylibre 2026-03-27 02:15:32,425 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3453818a3717aa65d3103 2026-03-27 02:15:32,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:15:32,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3453818a3717aa65d3103 HTTP/1.1" 200 None 2026-03-27 02:15:32,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:15:32,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086b1ce0c9bc43bccb63a HTTP/1.1" 200 None 2026-03-27 02:15:32,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:15:32,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3453818a3717aa65d3103 HTTP/1.1" 200 None 2026-03-27 02:15:32,478 - lava_callback - ERROR - Error processing callback for node 69c3453818a3717aa65d3103: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41770 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40514 - "GET / HTTP/1.0" 200 OK 2026-03-27 02:16:52,642 - lava_callback - INFO - Received callback for node 69c3458a18a3717aa65d310c INFO: 172.20.0.1:52874 - "POST /node/69c3458a18a3717aa65d310c HTTP/1.0" 202 Accepted 2026-03-27 02:16:52,648 - lava_callback - INFO - Starting async job submit for node 69c3458a18a3717aa65d310c 2026-03-27 02:16:52,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:16:52,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3458a18a3717aa65d310c HTTP/1.1" 200 None 2026-03-27 02:16:52,677 - lava_callback - DEBUG - Processing node 69c3458a18a3717aa65d310c: baseline-arm64-baylibre 2026-03-27 02:16:52,677 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3458a18a3717aa65d310c 2026-03-27 02:16:52,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:16:52,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3458a18a3717aa65d310c HTTP/1.1" 200 None 2026-03-27 02:16:52,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:16:52,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086b1ce0c9bc43bccb63a HTTP/1.1" 200 None 2026-03-27 02:16:52,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:16:52,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3458a18a3717aa65d310c HTTP/1.1" 200 None 2026-03-27 02:16:52,756 - lava_callback - ERROR - Error processing callback for node 69c3458a18a3717aa65d310c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57322 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50776 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50790 - "GET / HTTP/1.0" 200 OK 2026-03-27 02:18:24,456 - lava_callback - INFO - Received callback for node 69c345e118a3717aa65d3115 INFO: 172.20.0.1:40468 - "POST /node/69c345e118a3717aa65d3115 HTTP/1.0" 202 Accepted 2026-03-27 02:18:24,465 - lava_callback - INFO - Starting async job submit for node 69c345e118a3717aa65d3115 2026-03-27 02:18:24,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:18:24,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c345e118a3717aa65d3115 HTTP/1.1" 200 None 2026-03-27 02:18:24,482 - lava_callback - DEBUG - Processing node 69c345e118a3717aa65d3115: baseline-arm64-clabbe 2026-03-27 02:18:24,482 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c345e118a3717aa65d3115 2026-03-27 02:18:24,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:18:24,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c345e118a3717aa65d3115 HTTP/1.1" 200 None 2026-03-27 02:18:24,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:18:24,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086b1ce0c9bc43bccb63a HTTP/1.1" 200 None 2026-03-27 02:18:24,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:18:24,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c345e118a3717aa65d3115 HTTP/1.1" 200 None 2026-03-27 02:18:24,538 - lava_callback - ERROR - Error processing callback for node 69c345e118a3717aa65d3115: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38214 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46782 - "GET / HTTP/1.0" 200 OK 2026-03-27 02:19:44,544 - lava_callback - INFO - Received callback for node 69c3463318a3717aa65d311f INFO: 172.20.0.1:39328 - "POST /node/69c3463318a3717aa65d311f HTTP/1.0" 202 Accepted 2026-03-27 02:19:44,549 - lava_callback - INFO - Starting async job submit for node 69c3463318a3717aa65d311f 2026-03-27 02:19:44,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:19:44,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3463318a3717aa65d311f HTTP/1.1" 200 None 2026-03-27 02:19:44,566 - lava_callback - DEBUG - Processing node 69c3463318a3717aa65d311f: baseline-arm64-clabbe 2026-03-27 02:19:44,566 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3463318a3717aa65d311f 2026-03-27 02:19:44,567 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:19:44,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3463318a3717aa65d311f HTTP/1.1" 200 None 2026-03-27 02:19:44,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:19:44,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086b1ce0c9bc43bccb63a HTTP/1.1" 200 None 2026-03-27 02:19:44,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:19:44,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3463318a3717aa65d311f HTTP/1.1" 200 None 2026-03-27 02:19:44,623 - lava_callback - ERROR - Error processing callback for node 69c3463318a3717aa65d311f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41820 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58132 - "GET / HTTP/1.0" 200 OK 2026-03-27 02:21:04,540 - lava_callback - INFO - Received callback for node 69c3468618a3717aa65d3127 INFO: 172.20.0.1:43980 - "POST /node/69c3468618a3717aa65d3127 HTTP/1.0" 202 Accepted 2026-03-27 02:21:04,545 - lava_callback - INFO - Starting async job submit for node 69c3468618a3717aa65d3127 2026-03-27 02:21:04,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:21:04,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3468618a3717aa65d3127 HTTP/1.1" 200 None 2026-03-27 02:21:04,561 - lava_callback - DEBUG - Processing node 69c3468618a3717aa65d3127: baseline-arm64-clabbe 2026-03-27 02:21:04,561 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3468618a3717aa65d3127 2026-03-27 02:21:04,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:21:04,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3468618a3717aa65d3127 HTTP/1.1" 200 None 2026-03-27 02:21:04,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:21:04,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086b1ce0c9bc43bccb63a HTTP/1.1" 200 None 2026-03-27 02:21:04,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:21:04,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3468618a3717aa65d3127 HTTP/1.1" 200 None 2026-03-27 02:21:04,621 - lava_callback - ERROR - Error processing callback for node 69c3468618a3717aa65d3127: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60638 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44894 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44896 - "GET / HTTP/1.0" 200 OK 2026-03-27 02:22:24,635 - lava_callback - INFO - Received callback for node 69c346d818a3717aa65d312d INFO: 172.20.0.1:52212 - "POST /node/69c346d818a3717aa65d312d HTTP/1.0" 202 Accepted 2026-03-27 02:22:24,641 - lava_callback - INFO - Starting async job submit for node 69c346d818a3717aa65d312d 2026-03-27 02:22:24,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:22:24,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c346d818a3717aa65d312d HTTP/1.1" 200 None 2026-03-27 02:22:24,657 - lava_callback - DEBUG - Processing node 69c346d818a3717aa65d312d: baseline-arm64-clabbe 2026-03-27 02:22:24,657 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c346d818a3717aa65d312d 2026-03-27 02:22:24,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:22:24,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c346d818a3717aa65d312d HTTP/1.1" 200 None 2026-03-27 02:22:24,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:22:24,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086b1ce0c9bc43bccb63a HTTP/1.1" 200 None 2026-03-27 02:22:24,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:22:24,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c346d818a3717aa65d312d HTTP/1.1" 200 None 2026-03-27 02:22:24,707 - lava_callback - ERROR - Error processing callback for node 69c346d818a3717aa65d312d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41454 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58748 - "GET / HTTP/1.0" 200 OK 2026-03-27 02:23:44,825 - lava_callback - INFO - Received callback for node 69c3472a18a3717aa65d3137 INFO: 172.20.0.1:45404 - "POST /node/69c3472a18a3717aa65d3137 HTTP/1.0" 202 Accepted 2026-03-27 02:23:44,830 - lava_callback - INFO - Starting async job submit for node 69c3472a18a3717aa65d3137 2026-03-27 02:23:44,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:23:44,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3472a18a3717aa65d3137 HTTP/1.1" 200 None 2026-03-27 02:23:44,872 - lava_callback - DEBUG - Processing node 69c3472a18a3717aa65d3137: baseline-arm64-clabbe 2026-03-27 02:23:44,873 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3472a18a3717aa65d3137 2026-03-27 02:23:44,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:23:44,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3472a18a3717aa65d3137 HTTP/1.1" 200 None 2026-03-27 02:23:44,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:23:44,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086b1ce0c9bc43bccb63a HTTP/1.1" 200 None 2026-03-27 02:23:44,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:23:44,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3472a18a3717aa65d3137 HTTP/1.1" 200 None 2026-03-27 02:23:44,962 - lava_callback - ERROR - Error processing callback for node 69c3472a18a3717aa65d3137: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46098 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55058 - "GET / HTTP/1.0" 200 OK 2026-03-27 02:24:48,484 - lava_callback - INFO - Received callback for node 69c56a7babf66a8f4d9bc1f3 INFO: 172.20.0.1:55066 - "POST /node/69c56a7babf66a8f4d9bc1f3 HTTP/1.0" 202 Accepted 2026-03-27 02:24:48,492 - lava_callback - INFO - Starting async job submit for node 69c56a7babf66a8f4d9bc1f3 2026-03-27 02:24:48,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:24:48,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c56a7babf66a8f4d9bc1f3 HTTP/1.1" 200 None 2026-03-27 02:24:48,542 - lava_callback - DEBUG - Processing node 69c56a7babf66a8f4d9bc1f3: baseline-arm64 2026-03-27 02:24:49,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 02:24:49,184 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 02:24:49,186 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c56a7babf66a8f4d9bc1f3/log.txt.gz 2026-03-27 02:24:49,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 02:24:49,684 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 02:24:49,686 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c56a7babf66a8f4d9bc1f3/lava-logs.yaml 2026-03-27 02:24:49,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 02:24:49,915 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 02:24:49,917 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c56a7babf66a8f4d9bc1f3/lava_callback.json.gz 2026-03-27 02:24:49,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:24:49,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c56a7babf66a8f4d9bc1f3 HTTP/1.1" 200 None 2026-03-27 02:24:49,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:24:49,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-27 02:24:49,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:24:50,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c56a7babf66a8f4d9bc1f3 HTTP/1.1" 200 None 2026-03-27 02:24:50,091 - lava_callback - INFO - Completed processing callback for node 69c56a7babf66a8f4d9bc1f3 INFO: 172.20.0.1:45510 - "GET / HTTP/1.0" 200 OK 2026-03-27 02:25:01,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:25:01,221 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-27 02:25:04,731 - lava_callback - INFO - Received callback for node 69c3477918a3717aa65d3141 INFO: 172.20.0.1:36456 - "POST /node/69c3477918a3717aa65d3141 HTTP/1.0" 202 Accepted 2026-03-27 02:25:04,737 - lava_callback - INFO - Starting async job submit for node 69c3477918a3717aa65d3141 2026-03-27 02:25:04,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:25:04,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3477918a3717aa65d3141 HTTP/1.1" 200 None 2026-03-27 02:25:04,753 - lava_callback - DEBUG - Processing node 69c3477918a3717aa65d3141: baseline-arm64-clabbe 2026-03-27 02:25:04,754 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3477918a3717aa65d3141 2026-03-27 02:25:04,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:25:04,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3477918a3717aa65d3141 HTTP/1.1" 200 None 2026-03-27 02:25:04,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:25:04,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086b1ce0c9bc43bccb63a HTTP/1.1" 200 None 2026-03-27 02:25:04,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:25:04,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3477918a3717aa65d3141 HTTP/1.1" 200 None 2026-03-27 02:25:04,814 - lava_callback - ERROR - Error processing callback for node 69c3477918a3717aa65d3141: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44460 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33004 - "GET / HTTP/1.0" 200 OK 2026-03-27 02:25:37,378 - lava_callback - INFO - Received callback for node 69c5b893abf66a8f4d9bcce9 INFO: 172.20.0.1:33010 - "POST /node/69c5b893abf66a8f4d9bcce9 HTTP/1.0" 202 Accepted 2026-03-27 02:25:37,384 - lava_callback - INFO - Starting async job submit for node 69c5b893abf66a8f4d9bcce9 2026-03-27 02:25:37,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:25:37,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b893abf66a8f4d9bcce9 HTTP/1.1" 200 None 2026-03-27 02:25:37,422 - lava_callback - DEBUG - Processing node 69c5b893abf66a8f4d9bcce9: baseline-arm64-broonie 2026-03-27 02:25:37,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 02:25:37,910 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 02:25:37,912 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c5b893abf66a8f4d9bcce9/log.txt.gz 2026-03-27 02:25:38,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 02:25:38,222 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 02:25:38,224 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c5b893abf66a8f4d9bcce9/lava-logs.yaml 2026-03-27 02:25:38,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 02:25:38,387 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 02:25:38,389 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c5b893abf66a8f4d9bcce9/lava_callback.json.gz 2026-03-27 02:25:38,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:25:38,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b893abf66a8f4d9bcce9 HTTP/1.1" 200 None 2026-03-27 02:25:38,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:25:38,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-27 02:25:38,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:25:38,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5b893abf66a8f4d9bcce9 HTTP/1.1" 200 None 2026-03-27 02:25:38,472 - lava_callback - INFO - Completed processing callback for node 69c5b893abf66a8f4d9bcce9 2026-03-27 02:25:39,017 - lava_callback - INFO - Received callback for node 69c5d59a2cb62322ad9662e8 INFO: 172.20.0.1:33022 - "POST /node/69c5d59a2cb62322ad9662e8 HTTP/1.0" 202 Accepted 2026-03-27 02:25:39,026 - lava_callback - INFO - Starting async job submit for node 69c5d59a2cb62322ad9662e8 2026-03-27 02:25:39,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:25:39,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d59a2cb62322ad9662e8 HTTP/1.1" 200 None 2026-03-27 02:25:39,075 - lava_callback - DEBUG - Processing node 69c5d59a2cb62322ad9662e8: baseline-arm64-broonie 2026-03-27 02:25:39,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 02:25:39,405 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 02:25:39,407 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c5d59a2cb62322ad9662e8/log.txt.gz 2026-03-27 02:25:39,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 02:25:39,708 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 02:25:39,710 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c5d59a2cb62322ad9662e8/lava-logs.yaml 2026-03-27 02:25:39,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 02:25:39,844 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 02:25:39,846 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c5d59a2cb62322ad9662e8/lava_callback.json.gz 2026-03-27 02:25:39,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:25:39,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d59a2cb62322ad9662e8 HTTP/1.1" 200 None 2026-03-27 02:25:39,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:25:39,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-27 02:25:39,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:25:39,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d59a2cb62322ad9662e8 HTTP/1.1" 200 None 2026-03-27 02:25:39,928 - lava_callback - INFO - Completed processing callback for node 69c5d59a2cb62322ad9662e8 INFO: 172.20.0.1:48826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59028 - "GET / HTTP/1.0" 200 OK 2026-03-27 02:26:01,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:26:01,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-27 02:26:02,570 - lava_callback - INFO - Received callback for node 69c5b87aabf66a8f4d9bcc82 INFO: 172.20.0.1:54188 - "POST /node/69c5b87aabf66a8f4d9bcc82 HTTP/1.0" 202 Accepted 2026-03-27 02:26:02,577 - lava_callback - INFO - Starting async job submit for node 69c5b87aabf66a8f4d9bcc82 2026-03-27 02:26:02,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:26:02,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b87aabf66a8f4d9bcc82 HTTP/1.1" 200 None 2026-03-27 02:26:02,612 - lava_callback - DEBUG - Processing node 69c5b87aabf66a8f4d9bcc82: baseline-nfs-arm64 2026-03-27 02:26:03,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 02:26:03,353 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 02:26:03,355 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5b87aabf66a8f4d9bcc82/log.txt.gz 2026-03-27 02:26:03,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 02:26:03,837 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 02:26:03,839 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5b87aabf66a8f4d9bcc82/lava-logs.yaml 2026-03-27 02:26:03,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 02:26:03,998 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 02:26:04,000 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5b87aabf66a8f4d9bcc82/lava_callback.json.gz 2026-03-27 02:26:04,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:26:04,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b87aabf66a8f4d9bcc82 HTTP/1.1" 200 None 2026-03-27 02:26:04,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:26:04,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-27 02:26:04,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:26:04,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5b87aabf66a8f4d9bcc82 HTTP/1.1" 200 None 2026-03-27 02:26:04,076 - lava_callback - INFO - Completed processing callback for node 69c5b87aabf66a8f4d9bcc82 INFO: 172.20.0.1:54192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36898 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55694 - "GET / HTTP/1.0" 200 OK 2026-03-27 02:26:31,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:26:31,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:58898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53438 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35762 - "GET / HTTP/1.0" 200 OK 2026-03-27 02:27:37,714 - lava_callback - INFO - Received callback for node 69c5b89cabf66a8f4d9bcd18 INFO: 172.20.0.1:35766 - "POST /node/69c5b89cabf66a8f4d9bcd18 HTTP/1.0" 202 Accepted 2026-03-27 02:27:37,720 - lava_callback - INFO - Starting async job submit for node 69c5b89cabf66a8f4d9bcd18 2026-03-27 02:27:37,742 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:27:37,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b89cabf66a8f4d9bcd18 HTTP/1.1" 200 None 2026-03-27 02:27:37,757 - lava_callback - DEBUG - Processing node 69c5b89cabf66a8f4d9bcd18: baseline-arm64-broonie 2026-03-27 02:27:37,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 02:27:38,167 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 02:27:38,169 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c5b89cabf66a8f4d9bcd18/log.txt.gz 2026-03-27 02:27:38,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 02:27:38,533 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 02:27:38,535 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c5b89cabf66a8f4d9bcd18/lava-logs.yaml 2026-03-27 02:27:38,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 02:27:38,706 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 02:27:38,708 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c5b89cabf66a8f4d9bcd18/lava_callback.json.gz 2026-03-27 02:27:38,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:27:38,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b89cabf66a8f4d9bcd18 HTTP/1.1" 200 None 2026-03-27 02:27:38,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:27:38,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-27 02:27:38,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:27:38,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5b89cabf66a8f4d9bcd18 HTTP/1.1" 200 None 2026-03-27 02:27:38,793 - lava_callback - INFO - Completed processing callback for node 69c5b89cabf66a8f4d9bcd18 INFO: 172.20.0.1:53032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56180 - "GET / HTTP/1.0" 200 OK 2026-03-27 02:28:01,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:28:01,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:44222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47126 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40488 - "GET / HTTP/1.0" 200 OK 2026-03-27 02:28:37,403 - lava_callback - INFO - Received callback for node 69c56d68abf66a8f4d9bc29b INFO: 172.20.0.1:40490 - "POST /node/69c56d68abf66a8f4d9bc29b HTTP/1.0" 202 Accepted 2026-03-27 02:28:37,409 - lava_callback - INFO - Starting async job submit for node 69c56d68abf66a8f4d9bc29b 2026-03-27 02:28:37,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:28:37,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c56d68abf66a8f4d9bc29b HTTP/1.1" 200 None 2026-03-27 02:28:37,452 - lava_callback - DEBUG - Processing node 69c56d68abf66a8f4d9bc29b: baseline-nfs-arm64 2026-03-27 02:28:37,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 02:28:38,136 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 02:28:38,137 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c56d68abf66a8f4d9bc29b/log.txt.gz 2026-03-27 02:28:38,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 02:28:38,782 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 02:28:38,784 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c56d68abf66a8f4d9bc29b/lava-logs.yaml 2026-03-27 02:28:38,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 02:28:38,972 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 02:28:38,974 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c56d68abf66a8f4d9bc29b/lava_callback.json.gz 2026-03-27 02:28:39,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:28:39,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c56d68abf66a8f4d9bc29b HTTP/1.1" 200 None 2026-03-27 02:28:39,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:28:39,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-27 02:28:39,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:28:39,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c56d68abf66a8f4d9bc29b HTTP/1.1" 200 None 2026-03-27 02:28:39,119 - lava_callback - INFO - Completed processing callback for node 69c56d68abf66a8f4d9bc29b INFO: 172.20.0.1:48140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40288 - "GET / HTTP/1.0" 200 OK 2026-03-27 02:28:57,457 - lava_callback - INFO - Received callback for node 69c5d5ad2cb62322ad966341 INFO: 172.20.0.1:40304 - "POST /node/69c5d5ad2cb62322ad966341 HTTP/1.0" 202 Accepted 2026-03-27 02:28:57,463 - lava_callback - INFO - Starting async job submit for node 69c5d5ad2cb62322ad966341 2026-03-27 02:28:57,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:28:57,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5ad2cb62322ad966341 HTTP/1.1" 200 None 2026-03-27 02:28:57,507 - lava_callback - DEBUG - Processing node 69c5d5ad2cb62322ad966341: baseline-arm64-broonie 2026-03-27 02:28:57,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 02:28:57,937 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 02:28:57,939 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c5d5ad2cb62322ad966341/log.txt.gz 2026-03-27 02:28:58,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 02:28:58,269 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 02:28:58,270 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c5d5ad2cb62322ad966341/lava-logs.yaml 2026-03-27 02:28:58,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 02:28:58,417 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 02:28:58,419 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c5d5ad2cb62322ad966341/lava_callback.json.gz 2026-03-27 02:28:58,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:28:58,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5ad2cb62322ad966341 HTTP/1.1" 200 None 2026-03-27 02:28:58,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:28:58,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-27 02:28:58,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:28:58,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d5ad2cb62322ad966341 HTTP/1.1" 200 None 2026-03-27 02:28:58,506 - lava_callback - INFO - Completed processing callback for node 69c5d5ad2cb62322ad966341 2026-03-27 02:29:01,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:29:01,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:49694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37822 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37628 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40306 - "GET / HTTP/1.0" 200 OK 2026-03-27 02:30:37,990 - lava_callback - INFO - Received callback for node 69c5e7ac2cb62322ad968ed7 INFO: 172.20.0.1:40312 - "POST /node/69c5e7ac2cb62322ad968ed7 HTTP/1.0" 202 Accepted 2026-03-27 02:30:37,997 - lava_callback - INFO - Starting async job submit for node 69c5e7ac2cb62322ad968ed7 2026-03-27 02:30:38,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:30:38,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5e7ac2cb62322ad968ed7 HTTP/1.1" 200 None 2026-03-27 02:30:38,071 - lava_callback - DEBUG - Processing node 69c5e7ac2cb62322ad968ed7: ltp-smoke 2026-03-27 02:30:38,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 02:30:38,715 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 02:30:38,717 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c5e7ac2cb62322ad968ed7/log.txt.gz 2026-03-27 02:30:39,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 02:30:39,243 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 02:30:39,245 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c5e7ac2cb62322ad968ed7/lava-logs.yaml 2026-03-27 02:30:39,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 02:30:39,411 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 02:30:39,413 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c5e7ac2cb62322ad968ed7/lava_callback.json.gz 2026-03-27 02:30:39,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:30:39,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5e7ac2cb62322ad968ed7 HTTP/1.1" 200 None 2026-03-27 02:30:39,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:30:39,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc312cb62322ad965dad HTTP/1.1" 200 None 2026-03-27 02:30:39,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:30:39,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5e7ac2cb62322ad968ed7 HTTP/1.1" 200 None 2026-03-27 02:30:39,746 - lava_callback - INFO - Completed processing callback for node 69c5e7ac2cb62322ad968ed7 INFO: 172.20.0.1:33670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50196 - "GET / HTTP/1.0" 200 OK 2026-03-27 02:31:01,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:31:01,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:54654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60656 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36838 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57548 - "GET / HTTP/1.0" 200 OK 2026-03-27 02:32:39,143 - lava_callback - INFO - Received callback for node 69c5d5412cb62322ad966234 INFO: 172.20.0.1:57554 - "POST /node/69c5d5412cb62322ad966234 HTTP/1.0" 202 Accepted 2026-03-27 02:32:39,149 - lava_callback - INFO - Starting async job submit for node 69c5d5412cb62322ad966234 2026-03-27 02:32:39,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:32:39,183 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5412cb62322ad966234 HTTP/1.1" 200 None 2026-03-27 02:32:39,184 - lava_callback - DEBUG - Processing node 69c5d5412cb62322ad966234: baseline-arm64 2026-03-27 02:32:39,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 02:32:39,724 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 02:32:39,726 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5d5412cb62322ad966234/log.txt.gz 2026-03-27 02:32:39,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 02:32:40,127 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 02:32:40,129 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5d5412cb62322ad966234/lava-logs.yaml 2026-03-27 02:32:40,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 02:32:40,269 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 02:32:40,271 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5d5412cb62322ad966234/lava_callback.json.gz 2026-03-27 02:32:40,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:32:40,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5412cb62322ad966234 HTTP/1.1" 200 None 2026-03-27 02:32:40,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:32:40,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-27 02:32:40,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:32:40,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d5412cb62322ad966234 HTTP/1.1" 200 None 2026-03-27 02:32:40,356 - lava_callback - INFO - Completed processing callback for node 69c5d5412cb62322ad966234 INFO: 172.20.0.1:42178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49068 - "GET / HTTP/1.0" 200 OK 2026-03-27 02:33:01,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:33:01,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:49234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33174 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46882 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33612 - "GET / HTTP/1.0" 200 OK 2026-03-27 02:35:00,971 - lava_callback - INFO - Received callback for node 69c5d5692cb62322ad966284 INFO: 172.20.0.1:60974 - "POST /node/69c5d5692cb62322ad966284 HTTP/1.0" 202 Accepted 2026-03-27 02:35:00,977 - lava_callback - INFO - Starting async job submit for node 69c5d5692cb62322ad966284 2026-03-27 02:35:00,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:35:01,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5692cb62322ad966284 HTTP/1.1" 200 None 2026-03-27 02:35:01,015 - lava_callback - DEBUG - Processing node 69c5d5692cb62322ad966284: baseline-nfs-arm64 2026-03-27 02:35:01,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 02:35:01,749 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 02:35:01,751 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5d5692cb62322ad966284/log.txt.gz 2026-03-27 02:35:01,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 02:35:02,270 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 02:35:02,272 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5d5692cb62322ad966284/lava-logs.yaml 2026-03-27 02:35:02,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 02:35:02,485 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 02:35:02,487 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5d5692cb62322ad966284/lava_callback.json.gz 2026-03-27 02:35:02,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:35:02,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5692cb62322ad966284 HTTP/1.1" 200 None 2026-03-27 02:35:02,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:35:02,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-27 02:35:02,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:35:02,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d5692cb62322ad966284 HTTP/1.1" 200 None 2026-03-27 02:35:02,598 - lava_callback - INFO - Completed processing callback for node 69c5d5692cb62322ad966284 INFO: 172.20.0.1:60976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36600 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48814 - "GET / HTTP/1.0" 200 OK 2026-03-27 02:35:31,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:35:31,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:49740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56888 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49034 - "GET / HTTP/1.0" 200 OK 2026-03-27 02:36:27,728 - lava_callback - INFO - Received callback for node 69c56e03abf66a8f4d9bc2cf INFO: 172.20.0.1:49036 - "POST /node/69c56e03abf66a8f4d9bc2cf HTTP/1.0" 202 Accepted 2026-03-27 02:36:27,734 - lava_callback - INFO - Starting async job submit for node 69c56e03abf66a8f4d9bc2cf 2026-03-27 02:36:27,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:36:27,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c56e03abf66a8f4d9bc2cf HTTP/1.1" 200 None 2026-03-27 02:36:27,772 - lava_callback - DEBUG - Processing node 69c56e03abf66a8f4d9bc2cf: baseline-arm64 2026-03-27 02:36:28,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 02:36:28,455 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 02:36:28,457 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c56e03abf66a8f4d9bc2cf/log.txt.gz 2026-03-27 02:36:28,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 02:36:28,904 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 02:36:28,906 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c56e03abf66a8f4d9bc2cf/lava-logs.yaml 2026-03-27 02:36:28,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 02:36:29,117 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 02:36:29,119 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c56e03abf66a8f4d9bc2cf/lava_callback.json.gz 2026-03-27 02:36:29,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:36:29,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c56e03abf66a8f4d9bc2cf HTTP/1.1" 200 None 2026-03-27 02:36:29,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:36:29,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-27 02:36:29,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:36:29,220 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c56e03abf66a8f4d9bc2cf HTTP/1.1" 200 None 2026-03-27 02:36:29,285 - lava_callback - INFO - Completed processing callback for node 69c56e03abf66a8f4d9bc2cf 2026-03-27 02:36:29,286 - lava_callback - INFO - Received callback for node 69c5e4e02cb62322ad968e0a INFO: 172.20.0.1:49046 - "POST /node/69c5e4e02cb62322ad968e0a HTTP/1.0" 202 Accepted 2026-03-27 02:36:29,322 - lava_callback - INFO - Starting async job submit for node 69c5e4e02cb62322ad968e0a 2026-03-27 02:36:29,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:36:29,389 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5e4e02cb62322ad968e0a HTTP/1.1" 200 None 2026-03-27 02:36:29,390 - lava_callback - DEBUG - Processing node 69c5e4e02cb62322ad968e0a: baseline-arm-pengutronix 2026-03-27 02:36:29,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 02:36:29,674 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 02:36:29,677 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c5e4e02cb62322ad968e0a/log.txt.gz 2026-03-27 02:36:29,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 02:36:29,944 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 02:36:29,946 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c5e4e02cb62322ad968e0a/lava-logs.yaml 2026-03-27 02:36:29,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 02:36:30,073 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 02:36:30,074 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c5e4e02cb62322ad968e0a/lava_callback.json.gz DEBUG: Setup failed for 69c5e4e02cb62322ad968e0a. Transitting job node result: incomplete -> fail 2026-03-27 02:36:30,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:36:30,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5e4e02cb62322ad968e0a HTTP/1.1" 200 None 2026-03-27 02:36:30,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:36:30,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-27 02:36:30,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:36:30,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5e4e02cb62322ad968e0a HTTP/1.1" 200 None 2026-03-27 02:36:30,237 - lava_callback - INFO - Completed processing callback for node 69c5e4e02cb62322ad968e0a 2026-03-27 02:36:31,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:36:31,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:47588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33650 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60150 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47316 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46776 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46790 - "GET /node/694bd893c19e09af58d497da%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:51426 - "GET /node/698cb9e784a45b903735a218%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:51436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39352 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39380 - "GET / HTTP/1.0" 200 OK 2026-03-27 02:41:17,621 - lava_callback - INFO - Received callback for node 69c56e5babf66a8f4d9bc2dd INFO: 172.20.0.1:39386 - "POST /node/69c56e5babf66a8f4d9bc2dd HTTP/1.0" 202 Accepted 2026-03-27 02:41:17,628 - lava_callback - INFO - Starting async job submit for node 69c56e5babf66a8f4d9bc2dd 2026-03-27 02:41:17,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:41:17,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c56e5babf66a8f4d9bc2dd HTTP/1.1" 200 None 2026-03-27 02:41:17,718 - lava_callback - DEBUG - Processing node 69c56e5babf66a8f4d9bc2dd: baseline-nfs-arm64 2026-03-27 02:41:18,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 02:41:18,427 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 02:41:18,429 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c56e5babf66a8f4d9bc2dd/log.txt.gz 2026-03-27 02:41:18,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 02:41:19,114 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 02:41:19,117 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c56e5babf66a8f4d9bc2dd/lava-logs.yaml 2026-03-27 02:41:19,129 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 02:41:19,291 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 02:41:19,293 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c56e5babf66a8f4d9bc2dd/lava_callback.json.gz 2026-03-27 02:41:19,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:41:19,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c56e5babf66a8f4d9bc2dd HTTP/1.1" 200 None 2026-03-27 02:41:19,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:41:19,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-27 02:41:19,351 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:41:19,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c56e5babf66a8f4d9bc2dd HTTP/1.1" 200 None 2026-03-27 02:41:19,457 - lava_callback - INFO - Completed processing callback for node 69c56e5babf66a8f4d9bc2dd INFO: 172.20.0.1:41442 - "GET / HTTP/1.0" 200 OK 2026-03-27 02:41:31,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:41:31,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:50120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51318 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43324 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34876 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56656 - "GET /node/6993c0cc3e9ba4930770d253%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:56670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57722 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46492 - "GET / HTTP/1.0" 200 OK 2026-03-27 02:45:41,008 - lava_callback - INFO - Received callback for node 69c5eab92cb62322ad968fbb INFO: 172.20.0.1:43052 - "POST /node/69c5eab92cb62322ad968fbb HTTP/1.0" 202 Accepted 2026-03-27 02:45:41,016 - lava_callback - INFO - Starting async job submit for node 69c5eab92cb62322ad968fbb 2026-03-27 02:45:41,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:45:41,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5eab92cb62322ad968fbb HTTP/1.1" 200 None 2026-03-27 02:45:41,111 - lava_callback - DEBUG - Processing node 69c5eab92cb62322ad968fbb: baseline-nfs-arm64 2026-03-27 02:45:42,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 02:45:42,398 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 02:45:42,399 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5eab92cb62322ad968fbb/log.txt.gz 2026-03-27 02:45:43,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 02:45:43,297 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 02:45:43,300 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5eab92cb62322ad968fbb/lava-logs.yaml 2026-03-27 02:45:43,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 02:45:43,503 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 02:45:43,505 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5eab92cb62322ad968fbb/lava_callback.json.gz DEBUG: Setup failed for 69c5eab92cb62322ad968fbb. Transitting job node result: incomplete -> fail 2026-03-27 02:45:43,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:45:43,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5eab92cb62322ad968fbb HTTP/1.1" 200 None 2026-03-27 02:45:43,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:45:43,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-27 02:45:43,625 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:45:43,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5eab92cb62322ad968fbb HTTP/1.1" 200 None 2026-03-27 02:45:43,742 - lava_callback - INFO - Completed processing callback for node 69c5eab92cb62322ad968fbb INFO: 172.20.0.1:43066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37504 - "GET / HTTP/1.0" 200 OK 2026-03-27 02:46:01,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:46:01,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:41574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33514 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50514 - "GET / HTTP/1.0" 200 OK 2026-03-27 02:46:34,040 - lava_callback - INFO - Received callback for node 69c34c7818a3717aa65d31a7 INFO: 172.20.0.1:38770 - "POST /node/69c34c7818a3717aa65d31a7 HTTP/1.0" 202 Accepted 2026-03-27 02:46:34,045 - lava_callback - INFO - Starting async job submit for node 69c34c7818a3717aa65d31a7 2026-03-27 02:46:34,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:46:34,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c34c7818a3717aa65d31a7 HTTP/1.1" 200 None 2026-03-27 02:46:34,061 - lava_callback - DEBUG - Processing node 69c34c7818a3717aa65d31a7: baseline-arm-baylibre 2026-03-27 02:46:34,061 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c34c7818a3717aa65d31a7 2026-03-27 02:46:34,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:46:34,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c34c7818a3717aa65d31a7 HTTP/1.1" 200 None 2026-03-27 02:46:34,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:46:34,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-27 02:46:34,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:46:34,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c34c7818a3717aa65d31a7 HTTP/1.1" 200 None 2026-03-27 02:46:34,115 - lava_callback - ERROR - Error processing callback for node 69c34c7818a3717aa65d31a7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55258 - "GET / HTTP/1.0" 200 OK 2026-03-27 02:46:53,990 - lava_callback - INFO - Received callback for node 69c34c9c18a3717aa65d31b5 INFO: 172.20.0.1:35422 - "POST /node/69c34c9c18a3717aa65d31b5 HTTP/1.0" 202 Accepted 2026-03-27 02:46:53,995 - lava_callback - INFO - Starting async job submit for node 69c34c9c18a3717aa65d31b5 2026-03-27 02:46:53,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:46:54,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c34c9c18a3717aa65d31b5 HTTP/1.1" 200 None 2026-03-27 02:46:54,026 - lava_callback - DEBUG - Processing node 69c34c9c18a3717aa65d31b5: baseline-arm-baylibre 2026-03-27 02:46:54,027 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c34c9c18a3717aa65d31b5 2026-03-27 02:46:54,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:46:54,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c34c9c18a3717aa65d31b5 HTTP/1.1" 200 None 2026-03-27 02:46:54,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:46:54,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-27 02:46:54,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:46:54,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c34c9c18a3717aa65d31b5 HTTP/1.1" 200 None 2026-03-27 02:46:54,080 - lava_callback - ERROR - Error processing callback for node 69c34c9c18a3717aa65d31b5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56244 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48226 - "GET / HTTP/1.0" 200 OK 2026-03-27 02:48:08,182 - lava_callback - INFO - Received callback for node 69c56ef6abf66a8f4d9bc2fd INFO: 172.20.0.1:48230 - "POST /node/69c56ef6abf66a8f4d9bc2fd HTTP/1.0" 202 Accepted 2026-03-27 02:48:08,188 - lava_callback - INFO - Starting async job submit for node 69c56ef6abf66a8f4d9bc2fd 2026-03-27 02:48:08,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:48:08,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c56ef6abf66a8f4d9bc2fd HTTP/1.1" 200 None 2026-03-27 02:48:08,227 - lava_callback - DEBUG - Processing node 69c56ef6abf66a8f4d9bc2fd: baseline-arm64 2026-03-27 02:48:08,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 02:48:09,057 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 02:48:09,059 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c56ef6abf66a8f4d9bc2fd/log.txt.gz 2026-03-27 02:48:09,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 02:48:09,539 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 02:48:09,542 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c56ef6abf66a8f4d9bc2fd/lava-logs.yaml 2026-03-27 02:48:09,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 02:48:09,723 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 02:48:09,725 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c56ef6abf66a8f4d9bc2fd/lava_callback.json.gz 2026-03-27 02:48:09,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:48:09,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c56ef6abf66a8f4d9bc2fd HTTP/1.1" 200 None 2026-03-27 02:48:09,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:48:09,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-27 02:48:09,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:48:09,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c56ef6abf66a8f4d9bc2fd HTTP/1.1" 200 None 2026-03-27 02:48:09,856 - lava_callback - INFO - Completed processing callback for node 69c56ef6abf66a8f4d9bc2fd INFO: 172.20.0.1:34862 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40490 - "GET / HTTP/1.0" 200 OK 2026-03-27 02:48:31,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:48:31,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:47702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33768 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42450 - "GET / HTTP/1.0" 200 OK 2026-03-27 02:49:34,247 - lava_callback - INFO - Received callback for node 69c34d3318a3717aa65d31d4 INFO: 172.20.0.1:40760 - "POST /node/69c34d3318a3717aa65d31d4 HTTP/1.0" 202 Accepted 2026-03-27 02:49:34,252 - lava_callback - INFO - Starting async job submit for node 69c34d3318a3717aa65d31d4 2026-03-27 02:49:34,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:49:34,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c34d3318a3717aa65d31d4 HTTP/1.1" 200 None 2026-03-27 02:49:34,277 - lava_callback - DEBUG - Processing node 69c34d3318a3717aa65d31d4: baseline-arm-baylibre 2026-03-27 02:49:34,277 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c34d3318a3717aa65d31d4 2026-03-27 02:49:34,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:49:34,292 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c34d3318a3717aa65d31d4 HTTP/1.1" 200 None 2026-03-27 02:49:34,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:49:34,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-27 02:49:34,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:49:34,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c34d3318a3717aa65d31d4 HTTP/1.1" 200 None 2026-03-27 02:49:34,330 - lava_callback - ERROR - Error processing callback for node 69c34d3318a3717aa65d31d4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48022 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45162 - "GET / HTTP/1.0" 200 OK 2026-03-27 02:50:54,261 - lava_callback - INFO - Received callback for node 69c34d8818a3717aa65d31ec INFO: 172.20.0.1:59914 - "POST /node/69c34d8818a3717aa65d31ec HTTP/1.0" 202 Accepted 2026-03-27 02:50:54,267 - lava_callback - INFO - Starting async job submit for node 69c34d8818a3717aa65d31ec 2026-03-27 02:50:54,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:50:54,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c34d8818a3717aa65d31ec HTTP/1.1" 200 None 2026-03-27 02:50:54,283 - lava_callback - DEBUG - Processing node 69c34d8818a3717aa65d31ec: baseline-arm-baylibre 2026-03-27 02:50:54,283 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c34d8818a3717aa65d31ec 2026-03-27 02:50:54,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:50:54,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c34d8818a3717aa65d31ec HTTP/1.1" 200 None 2026-03-27 02:50:54,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:50:54,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-27 02:50:54,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:50:54,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c34d8818a3717aa65d31ec HTTP/1.1" 200 None 2026-03-27 02:50:54,357 - lava_callback - ERROR - Error processing callback for node 69c34d8818a3717aa65d31ec: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35344 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44406 - "GET / HTTP/1.0" 200 OK 2026-03-27 02:51:34,237 - lava_callback - INFO - Received callback for node 69c34dac18a3717aa65d31f5 INFO: 172.20.0.1:41208 - "POST /node/69c34dac18a3717aa65d31f5 HTTP/1.0" 202 Accepted 2026-03-27 02:51:34,243 - lava_callback - INFO - Starting async job submit for node 69c34dac18a3717aa65d31f5 2026-03-27 02:51:34,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:51:34,258 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c34dac18a3717aa65d31f5 HTTP/1.1" 200 None 2026-03-27 02:51:34,258 - lava_callback - DEBUG - Processing node 69c34dac18a3717aa65d31f5: baseline-arm-baylibre 2026-03-27 02:51:34,259 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c34dac18a3717aa65d31f5 2026-03-27 02:51:34,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:51:34,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c34dac18a3717aa65d31f5 HTTP/1.1" 200 None 2026-03-27 02:51:34,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:51:34,286 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-27 02:51:34,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:51:34,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c34dac18a3717aa65d31f5 HTTP/1.1" 200 None 2026-03-27 02:51:34,314 - lava_callback - ERROR - Error processing callback for node 69c34dac18a3717aa65d31f5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42664 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51286 - "GET / HTTP/1.0" 200 OK 2026-03-27 02:53:14,416 - lava_callback - INFO - Received callback for node 69c34e0718a3717aa65d3209 INFO: 172.20.0.1:52664 - "POST /node/69c34e0718a3717aa65d3209 HTTP/1.0" 202 Accepted 2026-03-27 02:53:14,421 - lava_callback - INFO - Starting async job submit for node 69c34e0718a3717aa65d3209 2026-03-27 02:53:14,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:53:14,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c34e0718a3717aa65d3209 HTTP/1.1" 200 None 2026-03-27 02:53:14,438 - lava_callback - DEBUG - Processing node 69c34e0718a3717aa65d3209: baseline-arm-baylibre 2026-03-27 02:53:14,438 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c34e0718a3717aa65d3209 2026-03-27 02:53:14,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:53:14,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c34e0718a3717aa65d3209 HTTP/1.1" 200 None 2026-03-27 02:53:14,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:53:14,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-27 02:53:14,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:53:14,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c34e0718a3717aa65d3209 HTTP/1.1" 200 None 2026-03-27 02:53:14,494 - lava_callback - ERROR - Error processing callback for node 69c34e0718a3717aa65d3209: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52672 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51954 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35174 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35186 - "GET / HTTP/1.0" 200 OK 2026-03-27 02:55:26,053 - lava_callback - INFO - Received callback for node 69c34e9918a3717aa65d3236 INFO: 172.20.0.1:49194 - "POST /node/69c34e9918a3717aa65d3236 HTTP/1.0" 202 Accepted 2026-03-27 02:55:26,058 - lava_callback - INFO - Starting async job submit for node 69c34e9918a3717aa65d3236 2026-03-27 02:55:26,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:55:26,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c34e9918a3717aa65d3236 HTTP/1.1" 200 None 2026-03-27 02:55:26,076 - lava_callback - DEBUG - Processing node 69c34e9918a3717aa65d3236: baseline-arm-clabbe 2026-03-27 02:55:26,077 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c34e9918a3717aa65d3236 2026-03-27 02:55:26,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:55:26,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c34e9918a3717aa65d3236 HTTP/1.1" 200 None 2026-03-27 02:55:26,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:55:26,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-27 02:55:26,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:55:26,129 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c34e9918a3717aa65d3236 HTTP/1.1" 200 None 2026-03-27 02:55:26,134 - lava_callback - ERROR - Error processing callback for node 69c34e9918a3717aa65d3236: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51864 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55468 - "GET / HTTP/1.0" 200 OK 2026-03-27 02:56:46,078 - lava_callback - INFO - Received callback for node 69c34ee718a3717aa65d323d INFO: 172.20.0.1:42154 - "POST /node/69c34ee718a3717aa65d323d HTTP/1.0" 202 Accepted 2026-03-27 02:56:46,083 - lava_callback - INFO - Starting async job submit for node 69c34ee718a3717aa65d323d 2026-03-27 02:56:46,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:56:46,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c34ee718a3717aa65d323d HTTP/1.1" 200 None 2026-03-27 02:56:46,122 - lava_callback - DEBUG - Processing node 69c34ee718a3717aa65d323d: baseline-arm-clabbe 2026-03-27 02:56:46,122 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c34ee718a3717aa65d323d 2026-03-27 02:56:46,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:56:46,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c34ee718a3717aa65d323d HTTP/1.1" 200 None 2026-03-27 02:56:46,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:56:46,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-27 02:56:46,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:56:46,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c34ee718a3717aa65d323d HTTP/1.1" 200 None 2026-03-27 02:56:46,203 - lava_callback - ERROR - Error processing callback for node 69c34ee718a3717aa65d323d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33514 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33522 - "GET / HTTP/1.0" 200 OK 2026-03-27 02:57:26,053 - lava_callback - INFO - Received callback for node 69c34f0c18a3717aa65d3247 INFO: 172.20.0.1:37750 - "POST /node/69c34f0c18a3717aa65d3247 HTTP/1.0" 202 Accepted 2026-03-27 02:57:26,059 - lava_callback - INFO - Starting async job submit for node 69c34f0c18a3717aa65d3247 2026-03-27 02:57:26,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:57:26,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c34f0c18a3717aa65d3247 HTTP/1.1" 200 None 2026-03-27 02:57:26,084 - lava_callback - DEBUG - Processing node 69c34f0c18a3717aa65d3247: baseline-arm-clabbe 2026-03-27 02:57:26,084 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c34f0c18a3717aa65d3247 2026-03-27 02:57:26,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:57:26,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c34f0c18a3717aa65d3247 HTTP/1.1" 200 None 2026-03-27 02:57:26,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:57:26,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-27 02:57:26,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:57:26,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c34f0c18a3717aa65d3247 HTTP/1.1" 200 None 2026-03-27 02:57:26,176 - lava_callback - ERROR - Error processing callback for node 69c34f0c18a3717aa65d3247: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35292 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60896 - "GET / HTTP/1.0" 200 OK 2026-03-27 02:58:46,097 - lava_callback - INFO - Received callback for node 69c34f5e18a3717aa65d3252 INFO: 172.20.0.1:42794 - "POST /node/69c34f5e18a3717aa65d3252 HTTP/1.0" 202 Accepted 2026-03-27 02:58:46,103 - lava_callback - INFO - Starting async job submit for node 69c34f5e18a3717aa65d3252 2026-03-27 02:58:46,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:58:46,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c34f5e18a3717aa65d3252 HTTP/1.1" 200 None 2026-03-27 02:58:46,120 - lava_callback - DEBUG - Processing node 69c34f5e18a3717aa65d3252: baseline-arm-clabbe 2026-03-27 02:58:46,120 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c34f5e18a3717aa65d3252 2026-03-27 02:58:46,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:58:46,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c34f5e18a3717aa65d3252 HTTP/1.1" 200 None 2026-03-27 02:58:46,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:58:46,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-27 02:58:46,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:58:46,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c34f5e18a3717aa65d3252 HTTP/1.1" 200 None 2026-03-27 02:58:46,185 - lava_callback - ERROR - Error processing callback for node 69c34f5e18a3717aa65d3252: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60306 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60312 - "GET / HTTP/1.0" 200 OK 2026-03-27 02:59:26,103 - lava_callback - INFO - Received callback for node 69c34f8318a3717aa65d325c INFO: 172.20.0.1:56102 - "POST /node/69c34f8318a3717aa65d325c HTTP/1.0" 202 Accepted 2026-03-27 02:59:26,108 - lava_callback - INFO - Starting async job submit for node 69c34f8318a3717aa65d325c 2026-03-27 02:59:26,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:59:26,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c34f8318a3717aa65d325c HTTP/1.1" 200 None 2026-03-27 02:59:26,125 - lava_callback - DEBUG - Processing node 69c34f8318a3717aa65d325c: baseline-arm-clabbe 2026-03-27 02:59:26,125 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c34f8318a3717aa65d325c 2026-03-27 02:59:26,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:59:26,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c34f8318a3717aa65d325c HTTP/1.1" 200 None 2026-03-27 02:59:26,140 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:59:26,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-27 02:59:26,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 02:59:26,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c34f8318a3717aa65d325c HTTP/1.1" 200 None 2026-03-27 02:59:26,212 - lava_callback - ERROR - Error processing callback for node 69c34f8318a3717aa65d325c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43476 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53114 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54144 - "GET / HTTP/1.0" 200 OK 2026-03-27 03:02:09,988 - lava_callback - INFO - Received callback for node 69c5eeb02cb62322ad9690f0 INFO: 172.20.0.1:47832 - "POST /node/69c5eeb02cb62322ad9690f0 HTTP/1.0" 202 Accepted 2026-03-27 03:02:10,037 - lava_callback - INFO - Starting async job submit for node 69c5eeb02cb62322ad9690f0 2026-03-27 03:02:10,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 03:02:10,082 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5eeb02cb62322ad9690f0 HTTP/1.1" 200 None 2026-03-27 03:02:10,083 - lava_callback - DEBUG - Processing node 69c5eeb02cb62322ad9690f0: baseline-arm-pengutronix 2026-03-27 03:02:10,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 03:02:10,487 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 03:02:10,489 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c5eeb02cb62322ad9690f0/log.txt.gz 2026-03-27 03:02:10,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 03:02:10,747 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 03:02:10,749 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c5eeb02cb62322ad9690f0/lava-logs.yaml 2026-03-27 03:02:10,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 03:02:10,889 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 03:02:10,890 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c5eeb02cb62322ad9690f0/lava_callback.json.gz DEBUG: Setup failed for 69c5eeb02cb62322ad9690f0. Transitting job node result: incomplete -> fail 2026-03-27 03:02:10,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 03:02:10,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5eeb02cb62322ad9690f0 HTTP/1.1" 200 None 2026-03-27 03:02:10,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 03:02:10,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-27 03:02:10,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 03:02:11,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5eeb02cb62322ad9690f0 HTTP/1.1" 200 None 2026-03-27 03:02:11,066 - lava_callback - INFO - Completed processing callback for node 69c5eeb02cb62322ad9690f0 INFO: 172.20.0.1:47848 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52404 - "GET / HTTP/1.0" 200 OK 2026-03-27 03:02:31,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 03:02:31,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:33092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55310 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56762 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59774 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35302 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46898 - "GET / HTTP/1.0" 200 OK 2026-03-27 03:06:37,197 - lava_callback - INFO - Received callback for node 69c578d2abf66a8f4d9bc4af INFO: 172.20.0.1:46910 - "POST /node/69c578d2abf66a8f4d9bc4af HTTP/1.0" 202 Accepted 2026-03-27 03:06:37,204 - lava_callback - INFO - Starting async job submit for node 69c578d2abf66a8f4d9bc4af 2026-03-27 03:06:37,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 03:06:37,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c578d2abf66a8f4d9bc4af HTTP/1.1" 200 None 2026-03-27 03:06:37,246 - lava_callback - DEBUG - Processing node 69c578d2abf66a8f4d9bc4af: baseline-nfs-arm64 2026-03-27 03:06:37,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 03:06:37,925 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 03:06:37,926 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c578d2abf66a8f4d9bc4af/log.txt.gz 2026-03-27 03:06:38,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 03:06:38,401 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 03:06:38,403 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c578d2abf66a8f4d9bc4af/lava-logs.yaml 2026-03-27 03:06:38,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 03:06:38,586 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 03:06:38,588 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c578d2abf66a8f4d9bc4af/lava_callback.json.gz 2026-03-27 03:06:38,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 03:06:38,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c578d2abf66a8f4d9bc4af HTTP/1.1" 200 None 2026-03-27 03:06:38,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 03:06:38,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-27 03:06:38,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 03:06:38,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c578d2abf66a8f4d9bc4af HTTP/1.1" 200 None 2026-03-27 03:06:38,749 - lava_callback - INFO - Completed processing callback for node 69c578d2abf66a8f4d9bc4af INFO: 172.20.0.1:60744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52588 - "GET / HTTP/1.0" 200 OK 2026-03-27 03:07:01,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 03:07:01,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:43530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45542 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34470 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52704 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46370 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44762 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52230 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33134 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56834 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57950 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-27 03:15:15,555 - lava_callback - INFO - Received callback for node 69c3532e18a3717aa65d32c8 INFO: 172.20.0.1:57952 - "POST /node/69c3532e18a3717aa65d32c8 HTTP/1.0" 202 Accepted 2026-03-27 03:15:15,561 - lava_callback - INFO - Starting async job submit for node 69c3532e18a3717aa65d32c8 2026-03-27 03:15:15,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 03:15:15,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3532e18a3717aa65d32c8 HTTP/1.1" 200 None 2026-03-27 03:15:15,576 - lava_callback - DEBUG - Processing node 69c3532e18a3717aa65d32c8: baseline-arm64-baylibre 2026-03-27 03:15:15,576 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3532e18a3717aa65d32c8 2026-03-27 03:15:15,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 03:15:15,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3532e18a3717aa65d32c8 HTTP/1.1" 200 None 2026-03-27 03:15:15,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 03:15:15,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-27 03:15:15,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 03:15:15,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3532e18a3717aa65d32c8 HTTP/1.1" 200 None 2026-03-27 03:15:15,628 - lava_callback - ERROR - Error processing callback for node 69c3532e18a3717aa65d32c8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39728 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50546 - "GET / HTTP/1.0" 200 OK 2026-03-27 03:16:35,597 - lava_callback - INFO - Received callback for node 69c3537e18a3717aa65d32d1 INFO: 172.20.0.1:59682 - "POST /node/69c3537e18a3717aa65d32d1 HTTP/1.0" 202 Accepted 2026-03-27 03:16:35,603 - lava_callback - INFO - Starting async job submit for node 69c3537e18a3717aa65d32d1 2026-03-27 03:16:35,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 03:16:35,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3537e18a3717aa65d32d1 HTTP/1.1" 200 None 2026-03-27 03:16:35,622 - lava_callback - DEBUG - Processing node 69c3537e18a3717aa65d32d1: baseline-arm64-baylibre 2026-03-27 03:16:35,622 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3537e18a3717aa65d32d1 2026-03-27 03:16:35,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 03:16:35,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3537e18a3717aa65d32d1 HTTP/1.1" 200 None 2026-03-27 03:16:35,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 03:16:35,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-27 03:16:35,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 03:16:35,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3537e18a3717aa65d32d1 HTTP/1.1" 200 None 2026-03-27 03:16:35,705 - lava_callback - ERROR - Error processing callback for node 69c3537e18a3717aa65d32d1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44386 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56540 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53214 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49196 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59426 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48950 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45748 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45750 - "GET / HTTP/1.0" 200 OK 2026-03-27 03:23:15,931 - lava_callback - INFO - Received callback for node 69c3551318a3717aa65d32e4 INFO: 172.20.0.1:45762 - "POST /node/69c3551318a3717aa65d32e4 HTTP/1.0" 202 Accepted 2026-03-27 03:23:15,936 - lava_callback - INFO - Starting async job submit for node 69c3551318a3717aa65d32e4 2026-03-27 03:23:15,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 03:23:15,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3551318a3717aa65d32e4 HTTP/1.1" 200 None 2026-03-27 03:23:15,953 - lava_callback - DEBUG - Processing node 69c3551318a3717aa65d32e4: baseline-arm64-baylibre 2026-03-27 03:23:15,953 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3551318a3717aa65d32e4 2026-03-27 03:23:15,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 03:23:15,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3551318a3717aa65d32e4 HTTP/1.1" 200 None 2026-03-27 03:23:15,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 03:23:15,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-27 03:23:15,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 03:23:16,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3551318a3717aa65d32e4 HTTP/1.1" 200 None 2026-03-27 03:23:16,009 - lava_callback - ERROR - Error processing callback for node 69c3551318a3717aa65d32e4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43186 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37022 - "GET / HTTP/1.0" 200 OK 2026-03-27 03:24:35,967 - lava_callback - INFO - Received callback for node 69c3556518a3717aa65d32f2 INFO: 172.20.0.1:40938 - "POST /node/69c3556518a3717aa65d32f2 HTTP/1.0" 202 Accepted 2026-03-27 03:24:35,972 - lava_callback - INFO - Starting async job submit for node 69c3556518a3717aa65d32f2 2026-03-27 03:24:35,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 03:24:35,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3556518a3717aa65d32f2 HTTP/1.1" 200 None 2026-03-27 03:24:35,988 - lava_callback - DEBUG - Processing node 69c3556518a3717aa65d32f2: baseline-arm64-baylibre 2026-03-27 03:24:35,988 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3556518a3717aa65d32f2 2026-03-27 03:24:35,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 03:24:36,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3556518a3717aa65d32f2 HTTP/1.1" 200 None 2026-03-27 03:24:36,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 03:24:36,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-27 03:24:36,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 03:24:36,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3556518a3717aa65d32f2 HTTP/1.1" 200 None 2026-03-27 03:24:36,048 - lava_callback - ERROR - Error processing callback for node 69c3556518a3717aa65d32f2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45976 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55968 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40562 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36108 - "GET / HTTP/1.0" 200 OK 2026-03-27 03:27:47,286 - lava_callback - INFO - Received callback for node 69c3561d18a3717aa65d3309 INFO: 172.20.0.1:36122 - "POST /node/69c3561d18a3717aa65d3309 HTTP/1.0" 202 Accepted 2026-03-27 03:27:47,292 - lava_callback - INFO - Starting async job submit for node 69c3561d18a3717aa65d3309 2026-03-27 03:27:47,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 03:27:47,307 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3561d18a3717aa65d3309 HTTP/1.1" 200 None 2026-03-27 03:27:47,308 - lava_callback - DEBUG - Processing node 69c3561d18a3717aa65d3309: baseline-arm64-clabbe 2026-03-27 03:27:47,309 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3561d18a3717aa65d3309 2026-03-27 03:27:47,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 03:27:47,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3561d18a3717aa65d3309 HTTP/1.1" 200 None 2026-03-27 03:27:47,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 03:27:47,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-27 03:27:47,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 03:27:47,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3561d18a3717aa65d3309 HTTP/1.1" 200 None 2026-03-27 03:27:47,359 - lava_callback - ERROR - Error processing callback for node 69c3561d18a3717aa65d3309: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35866 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35768 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41496 - "GET / HTTP/1.0" 200 OK 2026-03-27 03:29:27,334 - lava_callback - INFO - Received callback for node 69c3568118a3717aa65d3318 INFO: 172.20.0.1:41510 - "POST /node/69c3568118a3717aa65d3318 HTTP/1.0" 202 Accepted 2026-03-27 03:29:27,339 - lava_callback - INFO - Starting async job submit for node 69c3568118a3717aa65d3318 2026-03-27 03:29:27,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 03:29:27,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3568118a3717aa65d3318 HTTP/1.1" 200 None 2026-03-27 03:29:27,355 - lava_callback - DEBUG - Processing node 69c3568118a3717aa65d3318: baseline-arm64-clabbe 2026-03-27 03:29:27,355 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3568118a3717aa65d3318 2026-03-27 03:29:27,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 03:29:27,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3568118a3717aa65d3318 HTTP/1.1" 200 None 2026-03-27 03:29:27,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 03:29:27,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-27 03:29:27,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 03:29:27,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3568118a3717aa65d3318 HTTP/1.1" 200 None 2026-03-27 03:29:27,407 - lava_callback - ERROR - Error processing callback for node 69c3568118a3717aa65d3318: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59230 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57864 - "GET / HTTP/1.0" 200 OK 2026-03-27 03:30:47,391 - lava_callback - INFO - Received callback for node 69c356d318a3717aa65d3326 INFO: 172.20.0.1:57880 - "POST /node/69c356d318a3717aa65d3326 HTTP/1.0" 202 Accepted 2026-03-27 03:30:47,396 - lava_callback - INFO - Starting async job submit for node 69c356d318a3717aa65d3326 2026-03-27 03:30:47,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 03:30:47,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c356d318a3717aa65d3326 HTTP/1.1" 200 None 2026-03-27 03:30:47,412 - lava_callback - DEBUG - Processing node 69c356d318a3717aa65d3326: baseline-arm64-clabbe 2026-03-27 03:30:47,412 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c356d318a3717aa65d3326 2026-03-27 03:30:47,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 03:30:47,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c356d318a3717aa65d3326 HTTP/1.1" 200 None 2026-03-27 03:30:47,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 03:30:47,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-27 03:30:47,448 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 03:30:47,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c356d318a3717aa65d3326 HTTP/1.1" 200 None 2026-03-27 03:30:47,475 - lava_callback - ERROR - Error processing callback for node 69c356d318a3717aa65d3326: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60666 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56442 - "GET / HTTP/1.0" 200 OK 2026-03-27 03:31:46,410 - lava_callback - INFO - Received callback for node 69c5a1c6abf66a8f4d9bc914 INFO: 172.20.0.1:58240 - "POST /node/69c5a1c6abf66a8f4d9bc914 HTTP/1.0" 202 Accepted 2026-03-27 03:31:46,416 - lava_callback - INFO - Starting async job submit for node 69c5a1c6abf66a8f4d9bc914 2026-03-27 03:31:46,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 03:31:46,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5a1c6abf66a8f4d9bc914 HTTP/1.1" 200 None 2026-03-27 03:31:46,457 - lava_callback - DEBUG - Processing node 69c5a1c6abf66a8f4d9bc914: baseline-arm64 INFO: 172.20.0.1:58242 - "GET / HTTP/1.0" 200 OK 2026-03-27 03:31:46,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 03:31:47,120 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 03:31:47,122 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5a1c6abf66a8f4d9bc914/log.txt.gz 2026-03-27 03:31:47,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 03:31:47,619 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 03:31:47,621 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5a1c6abf66a8f4d9bc914/lava-logs.yaml 2026-03-27 03:31:47,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 03:31:47,835 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 03:31:47,837 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5a1c6abf66a8f4d9bc914/lava_callback.json.gz 2026-03-27 03:31:47,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 03:31:47,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5a1c6abf66a8f4d9bc914 HTTP/1.1" 200 None 2026-03-27 03:31:47,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 03:31:47,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-27 03:31:47,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 03:31:47,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5a1c6abf66a8f4d9bc914 HTTP/1.1" 200 None 2026-03-27 03:31:47,983 - lava_callback - INFO - Completed processing callback for node 69c5a1c6abf66a8f4d9bc914 INFO: 172.20.0.1:33048 - "GET / HTTP/1.0" 200 OK 2026-03-27 03:32:01,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 03:32:01,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:59930 - "GET / HTTP/1.0" 200 OK 2026-03-27 03:32:07,470 - lava_callback - INFO - Received callback for node 69c3572318a3717aa65d3330 INFO: 172.20.0.1:59944 - "POST /node/69c3572318a3717aa65d3330 HTTP/1.0" 202 Accepted 2026-03-27 03:32:07,476 - lava_callback - INFO - Starting async job submit for node 69c3572318a3717aa65d3330 2026-03-27 03:32:07,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 03:32:07,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3572318a3717aa65d3330 HTTP/1.1" 200 None 2026-03-27 03:32:07,515 - lava_callback - DEBUG - Processing node 69c3572318a3717aa65d3330: baseline-arm64-clabbe 2026-03-27 03:32:07,516 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3572318a3717aa65d3330 2026-03-27 03:32:07,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 03:32:07,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3572318a3717aa65d3330 HTTP/1.1" 200 None 2026-03-27 03:32:07,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 03:32:07,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-27 03:32:07,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 03:32:07,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3572318a3717aa65d3330 HTTP/1.1" 200 None 2026-03-27 03:32:07,579 - lava_callback - ERROR - Error processing callback for node 69c3572318a3717aa65d3330: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60888 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48342 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59568 - "GET / HTTP/1.0" 200 OK 2026-03-27 03:33:27,462 - lava_callback - INFO - Received callback for node 69c3577318a3717aa65d333e INFO: 172.20.0.1:59576 - "POST /node/69c3577318a3717aa65d333e HTTP/1.0" 202 Accepted 2026-03-27 03:33:27,470 - lava_callback - INFO - Starting async job submit for node 69c3577318a3717aa65d333e 2026-03-27 03:33:27,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 03:33:27,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3577318a3717aa65d333e HTTP/1.1" 200 None 2026-03-27 03:33:27,489 - lava_callback - DEBUG - Processing node 69c3577318a3717aa65d333e: baseline-arm64-clabbe 2026-03-27 03:33:27,489 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3577318a3717aa65d333e 2026-03-27 03:33:27,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 03:33:27,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3577318a3717aa65d333e HTTP/1.1" 200 None 2026-03-27 03:33:27,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 03:33:27,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-27 03:33:27,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 03:33:27,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3577318a3717aa65d333e HTTP/1.1" 200 None 2026-03-27 03:33:27,563 - lava_callback - ERROR - Error processing callback for node 69c3577318a3717aa65d333e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34662 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58614 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58634 - "GET / HTTP/1.0" 200 OK 2026-03-27 03:35:20,268 - lava_callback - INFO - Received callback for node 69c5a21eabf66a8f4d9bc922 INFO: 172.20.0.1:48670 - "POST /node/69c5a21eabf66a8f4d9bc922 HTTP/1.0" 202 Accepted 2026-03-27 03:35:20,275 - lava_callback - INFO - Starting async job submit for node 69c5a21eabf66a8f4d9bc922 2026-03-27 03:35:20,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 03:35:20,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5a21eabf66a8f4d9bc922 HTTP/1.1" 200 None 2026-03-27 03:35:20,316 - lava_callback - DEBUG - Processing node 69c5a21eabf66a8f4d9bc922: baseline-nfs-arm64 2026-03-27 03:35:20,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 03:35:21,005 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 03:35:21,007 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5a21eabf66a8f4d9bc922/log.txt.gz 2026-03-27 03:35:21,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 03:35:21,679 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 03:35:21,681 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5a21eabf66a8f4d9bc922/lava-logs.yaml 2026-03-27 03:35:21,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 03:35:21,866 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 03:35:21,867 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5a21eabf66a8f4d9bc922/lava_callback.json.gz 2026-03-27 03:35:21,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 03:35:21,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5a21eabf66a8f4d9bc922 HTTP/1.1" 200 None 2026-03-27 03:35:21,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 03:35:21,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-27 03:35:21,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 03:35:21,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5a21eabf66a8f4d9bc922 HTTP/1.1" 200 None 2026-03-27 03:35:22,018 - lava_callback - INFO - Completed processing callback for node 69c5a21eabf66a8f4d9bc922 INFO: 172.20.0.1:48672 - "GET / HTTP/1.0" 200 OK 2026-03-27 03:35:27,550 - lava_callback - INFO - Received callback for node 69c357eb18a3717aa65d334b INFO: 172.20.0.1:48678 - "POST /node/69c357eb18a3717aa65d334b HTTP/1.0" 202 Accepted 2026-03-27 03:35:27,555 - lava_callback - INFO - Starting async job submit for node 69c357eb18a3717aa65d334b 2026-03-27 03:35:27,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 03:35:27,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c357eb18a3717aa65d334b HTTP/1.1" 200 None 2026-03-27 03:35:27,572 - lava_callback - DEBUG - Processing node 69c357eb18a3717aa65d334b: baseline-arm64-clabbe 2026-03-27 03:35:27,573 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c357eb18a3717aa65d334b 2026-03-27 03:35:27,573 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 03:35:27,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c357eb18a3717aa65d334b HTTP/1.1" 200 None 2026-03-27 03:35:27,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 03:35:27,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-27 03:35:27,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 03:35:27,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c357eb18a3717aa65d334b HTTP/1.1" 200 None 2026-03-27 03:35:27,634 - lava_callback - ERROR - Error processing callback for node 69c357eb18a3717aa65d334b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-27 03:35:31,599 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 03:35:31,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:45828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39544 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47466 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46488 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48980 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55136 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40348 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50070 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34392 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42256 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57310 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34736 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40986 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57666 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56086 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51138 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49212 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39642 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40994 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45774 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37332 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33452 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51014 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41082 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34254 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50406 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38222 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38346 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46884 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39348 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52504 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33316 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39840 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54258 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57872 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35212 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47188 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46628 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60844 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47314 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38492 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37492 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39332 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56296 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46900 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50564 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48088 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54526 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52438 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44550 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47880 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57600 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43818 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38986 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33440 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50216 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40640 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39622 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60898 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45474 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59208 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35460 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43622 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38454 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35040 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36050 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60934 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48940 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41346 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48748 - "GET / HTTP/1.0" 200 OK 2026-03-27 04:43:40,828 - lava_callback - INFO - Received callback for node 69c367f018a3717aa65d33b6 INFO: 172.20.0.1:42756 - "POST /node/69c367f018a3717aa65d33b6 HTTP/1.0" 202 Accepted 2026-03-27 04:43:40,834 - lava_callback - INFO - Starting async job submit for node 69c367f018a3717aa65d33b6 2026-03-27 04:43:40,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:43:40,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c367f018a3717aa65d33b6 HTTP/1.1" 200 None 2026-03-27 04:43:40,853 - lava_callback - DEBUG - Processing node 69c367f018a3717aa65d33b6: baseline-arm-baylibre 2026-03-27 04:43:40,853 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c367f018a3717aa65d33b6 2026-03-27 04:43:40,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:43:40,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c367f018a3717aa65d33b6 HTTP/1.1" 200 None 2026-03-27 04:43:40,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:43:40,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-27 04:43:40,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:43:40,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c367f018a3717aa65d33b6 HTTP/1.1" 200 None 2026-03-27 04:43:40,910 - lava_callback - ERROR - Error processing callback for node 69c367f018a3717aa65d33b6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32906 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32936 - "GET / HTTP/1.0" 200 OK 2026-03-27 04:44:20,435 - lava_callback - INFO - Received callback for node 69c3681418a3717aa65d33c1 INFO: 172.20.0.1:36786 - "POST /node/69c3681418a3717aa65d33c1 HTTP/1.0" 202 Accepted 2026-03-27 04:44:20,441 - lava_callback - INFO - Starting async job submit for node 69c3681418a3717aa65d33c1 2026-03-27 04:44:20,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:44:20,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3681418a3717aa65d33c1 HTTP/1.1" 200 None 2026-03-27 04:44:20,457 - lava_callback - DEBUG - Processing node 69c3681418a3717aa65d33c1: baseline-arm-baylibre 2026-03-27 04:44:20,457 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3681418a3717aa65d33c1 2026-03-27 04:44:20,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:44:20,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3681418a3717aa65d33c1 HTTP/1.1" 200 None 2026-03-27 04:44:20,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:44:20,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-27 04:44:20,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:44:20,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3681418a3717aa65d33c1 HTTP/1.1" 200 None 2026-03-27 04:44:20,557 - lava_callback - ERROR - Error processing callback for node 69c3681418a3717aa65d33c1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55320 - "GET / HTTP/1.0" 200 OK 2026-03-27 04:45:00,490 - lava_callback - INFO - Received callback for node 69c3684618a3717aa65d33d0 INFO: 172.20.0.1:44076 - "POST /node/69c3684618a3717aa65d33d0 HTTP/1.0" 202 Accepted 2026-03-27 04:45:00,496 - lava_callback - INFO - Starting async job submit for node 69c3684618a3717aa65d33d0 2026-03-27 04:45:00,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:45:00,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3684618a3717aa65d33d0 HTTP/1.1" 200 None 2026-03-27 04:45:00,513 - lava_callback - DEBUG - Processing node 69c3684618a3717aa65d33d0: baseline-arm-baylibre 2026-03-27 04:45:00,514 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3684618a3717aa65d33d0 2026-03-27 04:45:00,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:45:00,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3684618a3717aa65d33d0 HTTP/1.1" 200 None 2026-03-27 04:45:00,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:45:00,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-27 04:45:00,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:45:00,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3684618a3717aa65d33d0 HTTP/1.1" 200 None 2026-03-27 04:45:00,573 - lava_callback - ERROR - Error processing callback for node 69c3684618a3717aa65d33d0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44628 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44742 - "GET / HTTP/1.0" 200 OK 2026-03-27 04:45:40,492 - lava_callback - INFO - Received callback for node 69c3686a18a3717aa65d33da INFO: 172.20.0.1:54866 - "POST /node/69c3686a18a3717aa65d33da HTTP/1.0" 202 Accepted 2026-03-27 04:45:40,498 - lava_callback - INFO - Starting async job submit for node 69c3686a18a3717aa65d33da 2026-03-27 04:45:40,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:45:40,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3686a18a3717aa65d33da HTTP/1.1" 200 None 2026-03-27 04:45:40,517 - lava_callback - DEBUG - Processing node 69c3686a18a3717aa65d33da: baseline-arm-baylibre 2026-03-27 04:45:40,517 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3686a18a3717aa65d33da 2026-03-27 04:45:40,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:45:40,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3686a18a3717aa65d33da HTTP/1.1" 200 None 2026-03-27 04:45:40,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:45:40,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-27 04:45:40,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:45:40,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3686a18a3717aa65d33da HTTP/1.1" 200 None 2026-03-27 04:45:40,577 - lava_callback - ERROR - Error processing callback for node 69c3686a18a3717aa65d33da: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59706 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59732 - "GET / HTTP/1.0" 200 OK 2026-03-27 04:46:20,605 - lava_callback - INFO - Received callback for node 69c3688f18a3717aa65d33e5 INFO: 172.20.0.1:45822 - "POST /node/69c3688f18a3717aa65d33e5 HTTP/1.0" 202 Accepted 2026-03-27 04:46:20,611 - lava_callback - INFO - Starting async job submit for node 69c3688f18a3717aa65d33e5 2026-03-27 04:46:20,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:46:20,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3688f18a3717aa65d33e5 HTTP/1.1" 200 None 2026-03-27 04:46:20,626 - lava_callback - DEBUG - Processing node 69c3688f18a3717aa65d33e5: baseline-arm-baylibre 2026-03-27 04:46:20,626 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3688f18a3717aa65d33e5 2026-03-27 04:46:20,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:46:20,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3688f18a3717aa65d33e5 HTTP/1.1" 200 None 2026-03-27 04:46:20,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:46:20,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-27 04:46:20,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:46:20,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3688f18a3717aa65d33e5 HTTP/1.1" 200 None 2026-03-27 04:46:20,693 - lava_callback - ERROR - Error processing callback for node 69c3688f18a3717aa65d33e5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50914 - "GET / HTTP/1.0" 200 OK 2026-03-27 04:47:00,589 - lava_callback - INFO - Received callback for node 69c368c118a3717aa65d33f5 INFO: 172.20.0.1:39606 - "POST /node/69c368c118a3717aa65d33f5 HTTP/1.0" 202 Accepted 2026-03-27 04:47:00,595 - lava_callback - INFO - Starting async job submit for node 69c368c118a3717aa65d33f5 2026-03-27 04:47:00,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:47:00,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c368c118a3717aa65d33f5 HTTP/1.1" 200 None 2026-03-27 04:47:00,612 - lava_callback - DEBUG - Processing node 69c368c118a3717aa65d33f5: baseline-arm-baylibre 2026-03-27 04:47:00,612 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c368c118a3717aa65d33f5 2026-03-27 04:47:00,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:47:00,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c368c118a3717aa65d33f5 HTTP/1.1" 200 None 2026-03-27 04:47:00,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:47:00,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-27 04:47:00,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:47:00,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c368c118a3717aa65d33f5 HTTP/1.1" 200 None 2026-03-27 04:47:00,673 - lava_callback - ERROR - Error processing callback for node 69c368c118a3717aa65d33f5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53168 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34454 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52822 - "GET / HTTP/1.0" 200 OK 2026-03-27 04:49:10,836 - lava_callback - INFO - Received callback for node 69c3693b18a3717aa65d341c INFO: 172.20.0.1:51494 - "POST /node/69c3693b18a3717aa65d341c HTTP/1.0" 202 Accepted 2026-03-27 04:49:10,841 - lava_callback - INFO - Starting async job submit for node 69c3693b18a3717aa65d341c 2026-03-27 04:49:10,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:49:10,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3693b18a3717aa65d341c HTTP/1.1" 200 None 2026-03-27 04:49:10,857 - lava_callback - DEBUG - Processing node 69c3693b18a3717aa65d341c: baseline-arm-clabbe 2026-03-27 04:49:10,857 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3693b18a3717aa65d341c 2026-03-27 04:49:10,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:49:10,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3693b18a3717aa65d341c HTTP/1.1" 200 None 2026-03-27 04:49:10,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:49:10,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-27 04:49:10,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:49:10,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3693b18a3717aa65d341c HTTP/1.1" 200 None 2026-03-27 04:49:10,921 - lava_callback - ERROR - Error processing callback for node 69c3693b18a3717aa65d341c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51502 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34228 - "GET / HTTP/1.0" 200 OK 2026-03-27 04:49:50,797 - lava_callback - INFO - Received callback for node 69c3696318a3717aa65d3428 INFO: 172.20.0.1:40132 - "POST /node/69c3696318a3717aa65d3428 HTTP/1.0" 202 Accepted 2026-03-27 04:49:50,803 - lava_callback - INFO - Starting async job submit for node 69c3696318a3717aa65d3428 2026-03-27 04:49:50,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:49:50,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3696318a3717aa65d3428 HTTP/1.1" 200 None 2026-03-27 04:49:50,818 - lava_callback - DEBUG - Processing node 69c3696318a3717aa65d3428: baseline-arm-clabbe 2026-03-27 04:49:50,818 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3696318a3717aa65d3428 2026-03-27 04:49:50,819 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:49:50,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3696318a3717aa65d3428 HTTP/1.1" 200 None 2026-03-27 04:49:50,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:49:50,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-27 04:49:50,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:49:50,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3696318a3717aa65d3428 HTTP/1.1" 200 None 2026-03-27 04:49:50,881 - lava_callback - ERROR - Error processing callback for node 69c3696318a3717aa65d3428: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56188 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56212 - "GET / HTTP/1.0" 200 OK 2026-03-27 04:50:20,773 - lava_callback - INFO - Received callback for node 69c3698418a3717aa65d3436 INFO: 172.20.0.1:39244 - "POST /node/69c3698418a3717aa65d3436 HTTP/1.0" 202 Accepted 2026-03-27 04:50:20,779 - lava_callback - INFO - Starting async job submit for node 69c3698418a3717aa65d3436 2026-03-27 04:50:20,779 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:50:20,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3698418a3717aa65d3436 HTTP/1.1" 200 None 2026-03-27 04:50:20,794 - lava_callback - DEBUG - Processing node 69c3698418a3717aa65d3436: baseline-arm-baylibre 2026-03-27 04:50:20,794 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3698418a3717aa65d3436 2026-03-27 04:50:20,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:50:20,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3698418a3717aa65d3436 HTTP/1.1" 200 None 2026-03-27 04:50:20,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:50:20,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-27 04:50:20,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:50:20,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3698418a3717aa65d3436 HTTP/1.1" 200 None 2026-03-27 04:50:20,846 - lava_callback - ERROR - Error processing callback for node 69c3698418a3717aa65d3436: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57196 - "GET / HTTP/1.0" 200 OK 2026-03-27 04:51:10,813 - lava_callback - INFO - Received callback for node 69c369af18a3717aa65d344a INFO: 172.20.0.1:56200 - "POST /node/69c369af18a3717aa65d344a HTTP/1.0" 202 Accepted 2026-03-27 04:51:10,818 - lava_callback - INFO - Starting async job submit for node 69c369af18a3717aa65d344a 2026-03-27 04:51:10,819 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:51:10,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c369af18a3717aa65d344a HTTP/1.1" 200 None 2026-03-27 04:51:10,835 - lava_callback - DEBUG - Processing node 69c369af18a3717aa65d344a: baseline-arm-clabbe 2026-03-27 04:51:10,835 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c369af18a3717aa65d344a 2026-03-27 04:51:10,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:51:10,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c369af18a3717aa65d344a HTTP/1.1" 200 None 2026-03-27 04:51:10,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:51:10,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-27 04:51:10,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:51:10,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c369af18a3717aa65d344a HTTP/1.1" 200 None 2026-03-27 04:51:10,900 - lava_callback - ERROR - Error processing callback for node 69c369af18a3717aa65d344a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56208 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40666 - "GET / HTTP/1.0" 200 OK 2026-03-27 04:51:40,808 - lava_callback - INFO - Received callback for node 69c369d118a3717aa65d3455 INFO: 172.20.0.1:54328 - "POST /node/69c369d118a3717aa65d3455 HTTP/1.0" 202 Accepted 2026-03-27 04:51:40,813 - lava_callback - INFO - Starting async job submit for node 69c369d118a3717aa65d3455 2026-03-27 04:51:40,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:51:40,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c369d118a3717aa65d3455 HTTP/1.1" 200 None 2026-03-27 04:51:40,856 - lava_callback - DEBUG - Processing node 69c369d118a3717aa65d3455: baseline-arm-baylibre 2026-03-27 04:51:40,856 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c369d118a3717aa65d3455 2026-03-27 04:51:40,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:51:40,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c369d118a3717aa65d3455 HTTP/1.1" 200 None 2026-03-27 04:51:40,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:51:40,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-27 04:51:40,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:51:40,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c369d118a3717aa65d3455 HTTP/1.1" 200 None 2026-03-27 04:51:40,909 - lava_callback - ERROR - Error processing callback for node 69c369d118a3717aa65d3455: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38816 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42500 - "GET / HTTP/1.0" 200 OK 2026-03-27 04:52:30,872 - lava_callback - INFO - Received callback for node 69c369fc18a3717aa65d3463 INFO: 172.20.0.1:44518 - "POST /node/69c369fc18a3717aa65d3463 HTTP/1.0" 202 Accepted 2026-03-27 04:52:30,878 - lava_callback - INFO - Starting async job submit for node 69c369fc18a3717aa65d3463 2026-03-27 04:52:30,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:52:30,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c369fc18a3717aa65d3463 HTTP/1.1" 200 None 2026-03-27 04:52:30,894 - lava_callback - DEBUG - Processing node 69c369fc18a3717aa65d3463: baseline-arm-clabbe 2026-03-27 04:52:30,894 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c369fc18a3717aa65d3463 2026-03-27 04:52:30,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:52:30,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c369fc18a3717aa65d3463 HTTP/1.1" 200 None 2026-03-27 04:52:30,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:52:30,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-27 04:52:30,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:52:30,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c369fc18a3717aa65d3463 HTTP/1.1" 200 None 2026-03-27 04:52:30,958 - lava_callback - ERROR - Error processing callback for node 69c369fc18a3717aa65d3463: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51950 - "GET / HTTP/1.0" 200 OK 2026-03-27 04:53:10,857 - lava_callback - INFO - Received callback for node 69c36a2218a3717aa65d3471 INFO: 172.20.0.1:35294 - "POST /node/69c36a2218a3717aa65d3471 HTTP/1.0" 202 Accepted 2026-03-27 04:53:10,862 - lava_callback - INFO - Starting async job submit for node 69c36a2218a3717aa65d3471 2026-03-27 04:53:10,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:53:10,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c36a2218a3717aa65d3471 HTTP/1.1" 200 None 2026-03-27 04:53:10,879 - lava_callback - DEBUG - Processing node 69c36a2218a3717aa65d3471: baseline-arm-clabbe 2026-03-27 04:53:10,879 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c36a2218a3717aa65d3471 2026-03-27 04:53:10,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:53:10,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c36a2218a3717aa65d3471 HTTP/1.1" 200 None 2026-03-27 04:53:10,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:53:10,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-27 04:53:10,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:53:10,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c36a2218a3717aa65d3471 HTTP/1.1" 200 None 2026-03-27 04:53:10,941 - lava_callback - ERROR - Error processing callback for node 69c36a2218a3717aa65d3471: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35306 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49424 - "GET / HTTP/1.0" 200 OK 2026-03-27 04:53:40,284 - lava_callback - INFO - Received callback for node 69c5b873abf66a8f4d9bcc70 INFO: 172.20.0.1:47322 - "POST /node/69c5b873abf66a8f4d9bcc70 HTTP/1.0" 202 Accepted 2026-03-27 04:53:40,291 - lava_callback - INFO - Starting async job submit for node 69c5b873abf66a8f4d9bcc70 2026-03-27 04:53:40,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:53:40,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b873abf66a8f4d9bcc70 HTTP/1.1" 200 None 2026-03-27 04:53:40,330 - lava_callback - DEBUG - Processing node 69c5b873abf66a8f4d9bcc70: baseline-arm64 2026-03-27 04:53:40,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 04:53:40,968 - lava_callback - INFO - Received callback for node 69c36a4418a3717aa65d347e INFO: 172.20.0.1:47334 - "POST /node/69c36a4418a3717aa65d347e HTTP/1.0" 202 Accepted 2026-03-27 04:53:40,974 - lava_callback - INFO - Starting async job submit for node 69c36a4418a3717aa65d347e 2026-03-27 04:53:40,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:53:40,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c36a4418a3717aa65d347e HTTP/1.1" 200 None 2026-03-27 04:53:40,991 - lava_callback - DEBUG - Processing node 69c36a4418a3717aa65d347e: baseline-arm-baylibre 2026-03-27 04:53:40,991 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c36a4418a3717aa65d347e 2026-03-27 04:53:40,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:53:41,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c36a4418a3717aa65d347e HTTP/1.1" 200 None 2026-03-27 04:53:41,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:53:41,031 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 04:53:41,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-27 04:53:41,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:53:41,034 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5b873abf66a8f4d9bcc70/log.txt.gz 2026-03-27 04:53:41,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c36a4418a3717aa65d347e HTTP/1.1" 200 None 2026-03-27 04:53:41,155 - lava_callback - ERROR - Error processing callback for node 69c36a4418a3717aa65d347e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-27 04:53:41,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 04:53:41,615 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 04:53:41,617 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5b873abf66a8f4d9bcc70/lava-logs.yaml 2026-03-27 04:53:41,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 04:53:41,793 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 04:53:41,795 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5b873abf66a8f4d9bcc70/lava_callback.json.gz 2026-03-27 04:53:41,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:53:41,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b873abf66a8f4d9bcc70 HTTP/1.1" 200 None 2026-03-27 04:53:41,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:53:41,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-27 04:53:41,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:53:41,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5b873abf66a8f4d9bcc70 HTTP/1.1" 200 None 2026-03-27 04:53:41,938 - lava_callback - INFO - Completed processing callback for node 69c5b873abf66a8f4d9bcc70 INFO: 172.20.0.1:47342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53542 - "GET / HTTP/1.0" 200 OK 2026-03-27 04:54:01,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:54:01,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:35694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40984 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41002 - "GET / HTTP/1.0" 200 OK 2026-03-27 04:54:20,961 - lava_callback - INFO - Received callback for node 69c36a6a18a3717aa65d348a INFO: 172.20.0.1:39524 - "POST /node/69c36a6a18a3717aa65d348a HTTP/1.0" 202 Accepted 2026-03-27 04:54:20,967 - lava_callback - INFO - Starting async job submit for node 69c36a6a18a3717aa65d348a 2026-03-27 04:54:20,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:54:20,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c36a6a18a3717aa65d348a HTTP/1.1" 200 None 2026-03-27 04:54:20,984 - lava_callback - DEBUG - Processing node 69c36a6a18a3717aa65d348a: baseline-arm-baylibre 2026-03-27 04:54:20,984 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c36a6a18a3717aa65d348a 2026-03-27 04:54:20,985 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:54:21,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c36a6a18a3717aa65d348a HTTP/1.1" 200 None 2026-03-27 04:54:21,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:54:21,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-27 04:54:21,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:54:21,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c36a6a18a3717aa65d348a HTTP/1.1" 200 None 2026-03-27 04:54:21,105 - lava_callback - ERROR - Error processing callback for node 69c36a6a18a3717aa65d348a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54014 - "GET / HTTP/1.0" 200 OK 2026-03-27 04:54:40,027 - lava_callback - INFO - Received callback for node 69c5b884abf66a8f4d9bccbd INFO: 172.20.0.1:50240 - "POST /node/69c5b884abf66a8f4d9bccbd HTTP/1.0" 202 Accepted 2026-03-27 04:54:40,034 - lava_callback - INFO - Starting async job submit for node 69c5b884abf66a8f4d9bccbd 2026-03-27 04:54:40,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:54:40,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b884abf66a8f4d9bccbd HTTP/1.1" 200 None 2026-03-27 04:54:40,077 - lava_callback - DEBUG - Processing node 69c5b884abf66a8f4d9bccbd: baseline-nfs-arm64 2026-03-27 04:54:40,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 04:54:40,736 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 04:54:40,737 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5b884abf66a8f4d9bccbd/log.txt.gz 2026-03-27 04:54:40,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 04:54:41,217 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 04:54:41,219 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5b884abf66a8f4d9bccbd/lava-logs.yaml 2026-03-27 04:54:41,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 04:54:41,545 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 04:54:41,547 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5b884abf66a8f4d9bccbd/lava_callback.json.gz 2026-03-27 04:54:41,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:54:41,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b884abf66a8f4d9bccbd HTTP/1.1" 200 None 2026-03-27 04:54:41,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:54:41,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-27 04:54:41,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:54:41,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5b884abf66a8f4d9bccbd HTTP/1.1" 200 None 2026-03-27 04:54:41,704 - lava_callback - INFO - Completed processing callback for node 69c5b884abf66a8f4d9bccbd INFO: 172.20.0.1:50254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41472 - "GET / HTTP/1.0" 200 OK 2026-03-27 04:55:01,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:55:01,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:57362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34292 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35476 - "GET / HTTP/1.0" 200 OK 2026-03-27 04:56:01,099 - lava_callback - INFO - Received callback for node 69c36acf18a3717aa65d349e INFO: 172.20.0.1:50680 - "POST /node/69c36acf18a3717aa65d349e HTTP/1.0" 202 Accepted 2026-03-27 04:56:01,104 - lava_callback - INFO - Starting async job submit for node 69c36acf18a3717aa65d349e 2026-03-27 04:56:01,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:56:01,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c36acf18a3717aa65d349e HTTP/1.1" 200 None 2026-03-27 04:56:01,121 - lava_callback - DEBUG - Processing node 69c36acf18a3717aa65d349e: baseline-arm-baylibre 2026-03-27 04:56:01,121 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c36acf18a3717aa65d349e 2026-03-27 04:56:01,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:56:01,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c36acf18a3717aa65d349e HTTP/1.1" 200 None 2026-03-27 04:56:01,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:56:01,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-27 04:56:01,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:56:01,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c36acf18a3717aa65d349e HTTP/1.1" 200 None 2026-03-27 04:56:01,173 - lava_callback - ERROR - Error processing callback for node 69c36acf18a3717aa65d349e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48414 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32930 - "GET / HTTP/1.0" 200 OK 2026-03-27 04:56:41,077 - lava_callback - INFO - Received callback for node 69c36af718a3717aa65d34b3 INFO: 172.20.0.1:59010 - "POST /node/69c36af718a3717aa65d34b3 HTTP/1.0" 202 Accepted 2026-03-27 04:56:41,083 - lava_callback - INFO - Starting async job submit for node 69c36af718a3717aa65d34b3 2026-03-27 04:56:41,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:56:41,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c36af718a3717aa65d34b3 HTTP/1.1" 200 None 2026-03-27 04:56:41,101 - lava_callback - DEBUG - Processing node 69c36af718a3717aa65d34b3: baseline-arm-baylibre 2026-03-27 04:56:41,101 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c36af718a3717aa65d34b3 2026-03-27 04:56:41,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:56:41,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c36af718a3717aa65d34b3 HTTP/1.1" 200 None 2026-03-27 04:56:41,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:56:41,129 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-27 04:56:41,130 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:56:41,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c36af718a3717aa65d34b3 HTTP/1.1" 200 None 2026-03-27 04:56:41,152 - lava_callback - ERROR - Error processing callback for node 69c36af718a3717aa65d34b3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47042 - "GET / HTTP/1.0" 200 OK 2026-03-27 04:57:01,098 - lava_callback - INFO - Received callback for node 69c36b1818a3717aa65d34c0 INFO: 172.20.0.1:37682 - "POST /node/69c36b1818a3717aa65d34c0 HTTP/1.0" 202 Accepted 2026-03-27 04:57:01,104 - lava_callback - INFO - Starting async job submit for node 69c36b1818a3717aa65d34c0 2026-03-27 04:57:01,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:57:01,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c36b1818a3717aa65d34c0 HTTP/1.1" 200 None 2026-03-27 04:57:01,120 - lava_callback - DEBUG - Processing node 69c36b1818a3717aa65d34c0: baseline-arm-baylibre 2026-03-27 04:57:01,120 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c36b1818a3717aa65d34c0 2026-03-27 04:57:01,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:57:01,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c36b1818a3717aa65d34c0 HTTP/1.1" 200 None 2026-03-27 04:57:01,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:57:01,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-27 04:57:01,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:57:01,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c36b1818a3717aa65d34c0 HTTP/1.1" 200 None 2026-03-27 04:57:01,180 - lava_callback - ERROR - Error processing callback for node 69c36b1818a3717aa65d34c0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57192 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47156 - "GET / HTTP/1.0" 200 OK 2026-03-27 04:57:41,205 - lava_callback - INFO - Received callback for node 69c36b3d18a3717aa65d34ce INFO: 172.20.0.1:37436 - "POST /node/69c36b3d18a3717aa65d34ce HTTP/1.0" 202 Accepted 2026-03-27 04:57:41,211 - lava_callback - INFO - Starting async job submit for node 69c36b3d18a3717aa65d34ce 2026-03-27 04:57:41,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:57:41,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c36b3d18a3717aa65d34ce HTTP/1.1" 200 None 2026-03-27 04:57:41,227 - lava_callback - DEBUG - Processing node 69c36b3d18a3717aa65d34ce: baseline-arm-baylibre 2026-03-27 04:57:41,227 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c36b3d18a3717aa65d34ce 2026-03-27 04:57:41,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:57:41,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c36b3d18a3717aa65d34ce HTTP/1.1" 200 None 2026-03-27 04:57:41,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:57:41,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-27 04:57:41,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:57:41,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c36b3d18a3717aa65d34ce HTTP/1.1" 200 None 2026-03-27 04:57:41,290 - lava_callback - ERROR - Error processing callback for node 69c36b3d18a3717aa65d34ce: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58328 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58354 - "GET / HTTP/1.0" 200 OK 2026-03-27 04:58:21,152 - lava_callback - INFO - Received callback for node 69c36b6218a3717aa65d34dc INFO: 172.20.0.1:54874 - "POST /node/69c36b6218a3717aa65d34dc HTTP/1.0" 202 Accepted 2026-03-27 04:58:21,157 - lava_callback - INFO - Starting async job submit for node 69c36b6218a3717aa65d34dc 2026-03-27 04:58:21,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:58:21,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c36b6218a3717aa65d34dc HTTP/1.1" 200 None 2026-03-27 04:58:21,174 - lava_callback - DEBUG - Processing node 69c36b6218a3717aa65d34dc: baseline-arm-baylibre 2026-03-27 04:58:21,174 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c36b6218a3717aa65d34dc 2026-03-27 04:58:21,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:58:21,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c36b6218a3717aa65d34dc HTTP/1.1" 200 None 2026-03-27 04:58:21,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:58:21,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-27 04:58:21,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:58:21,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c36b6218a3717aa65d34dc HTTP/1.1" 200 None 2026-03-27 04:58:21,231 - lava_callback - ERROR - Error processing callback for node 69c36b6218a3717aa65d34dc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55958 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55076 - "GET / HTTP/1.0" 200 OK 2026-03-27 04:59:31,148 - lava_callback - INFO - Received callback for node 69c36bae18a3717aa65d34f5 INFO: 172.20.0.1:44596 - "POST /node/69c36bae18a3717aa65d34f5 HTTP/1.0" 202 Accepted 2026-03-27 04:59:31,154 - lava_callback - INFO - Starting async job submit for node 69c36bae18a3717aa65d34f5 2026-03-27 04:59:31,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:59:31,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c36bae18a3717aa65d34f5 HTTP/1.1" 200 None 2026-03-27 04:59:31,170 - lava_callback - DEBUG - Processing node 69c36bae18a3717aa65d34f5: baseline-arm-clabbe 2026-03-27 04:59:31,170 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c36bae18a3717aa65d34f5 2026-03-27 04:59:31,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:59:31,187 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c36bae18a3717aa65d34f5 HTTP/1.1" 200 None 2026-03-27 04:59:31,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:59:31,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-27 04:59:31,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 04:59:31,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c36bae18a3717aa65d34f5 HTTP/1.1" 200 None 2026-03-27 04:59:31,238 - lava_callback - ERROR - Error processing callback for node 69c36bae18a3717aa65d34f5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57718 - "GET / HTTP/1.0" 200 OK 2026-03-27 05:00:11,295 - lava_callback - INFO - Received callback for node 69c36bd318a3717aa65d34fd INFO: 172.20.0.1:46328 - "POST /node/69c36bd318a3717aa65d34fd HTTP/1.0" 202 Accepted 2026-03-27 05:00:11,301 - lava_callback - INFO - Starting async job submit for node 69c36bd318a3717aa65d34fd 2026-03-27 05:00:11,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:00:11,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c36bd318a3717aa65d34fd HTTP/1.1" 200 None 2026-03-27 05:00:11,317 - lava_callback - DEBUG - Processing node 69c36bd318a3717aa65d34fd: baseline-arm-clabbe 2026-03-27 05:00:11,317 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c36bd318a3717aa65d34fd 2026-03-27 05:00:11,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:00:11,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c36bd318a3717aa65d34fd HTTP/1.1" 200 None 2026-03-27 05:00:11,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:00:11,351 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-27 05:00:11,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:00:11,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c36bd318a3717aa65d34fd HTTP/1.1" 200 None 2026-03-27 05:00:11,383 - lava_callback - ERROR - Error processing callback for node 69c36bd318a3717aa65d34fd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46344 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36580 - "GET / HTTP/1.0" 200 OK 2026-03-27 05:00:41,335 - lava_callback - INFO - Received callback for node 69c36bf618a3717aa65d3507 INFO: 172.20.0.1:36318 - "POST /node/69c36bf618a3717aa65d3507 HTTP/1.0" 202 Accepted 2026-03-27 05:00:41,341 - lava_callback - INFO - Starting async job submit for node 69c36bf618a3717aa65d3507 2026-03-27 05:00:41,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:00:41,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c36bf618a3717aa65d3507 HTTP/1.1" 200 None 2026-03-27 05:00:41,358 - lava_callback - DEBUG - Processing node 69c36bf618a3717aa65d3507: baseline-arm-baylibre 2026-03-27 05:00:41,359 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c36bf618a3717aa65d3507 2026-03-27 05:00:41,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:00:41,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c36bf618a3717aa65d3507 HTTP/1.1" 200 None 2026-03-27 05:00:41,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:00:41,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-27 05:00:41,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:00:41,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c36bf618a3717aa65d3507 HTTP/1.1" 200 None 2026-03-27 05:00:41,416 - lava_callback - ERROR - Error processing callback for node 69c36bf618a3717aa65d3507: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36800 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36830 - "GET / HTTP/1.0" 200 OK 2026-03-27 05:01:21,320 - lava_callback - INFO - Received callback for node 69c36c1a18a3717aa65d3515 INFO: 172.20.0.1:50312 - "POST /node/69c36c1a18a3717aa65d3515 HTTP/1.0" 202 Accepted 2026-03-27 05:01:21,326 - lava_callback - INFO - Starting async job submit for node 69c36c1a18a3717aa65d3515 2026-03-27 05:01:21,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:01:21,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c36c1a18a3717aa65d3515 HTTP/1.1" 200 None 2026-03-27 05:01:21,342 - lava_callback - DEBUG - Processing node 69c36c1a18a3717aa65d3515: baseline-arm-baylibre 2026-03-27 05:01:21,342 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c36c1a18a3717aa65d3515 2026-03-27 05:01:21,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:01:21,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c36c1a18a3717aa65d3515 HTTP/1.1" 200 None 2026-03-27 05:01:21,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:01:21,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-27 05:01:21,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:01:21,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c36c1a18a3717aa65d3515 HTTP/1.1" 200 None 2026-03-27 05:01:21,421 - lava_callback - ERROR - Error processing callback for node 69c36c1a18a3717aa65d3515: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54488 - "GET / HTTP/1.0" 200 OK 2026-03-27 05:02:11,238 - lava_callback - INFO - Received callback for node 69c36c3f18a3717aa65d3522 INFO: 172.20.0.1:54582 - "POST /node/69c36c3f18a3717aa65d3522 HTTP/1.0" 202 Accepted 2026-03-27 05:02:11,244 - lava_callback - INFO - Starting async job submit for node 69c36c3f18a3717aa65d3522 2026-03-27 05:02:11,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:02:11,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c36c3f18a3717aa65d3522 HTTP/1.1" 200 None 2026-03-27 05:02:11,261 - lava_callback - DEBUG - Processing node 69c36c3f18a3717aa65d3522: baseline-arm-clabbe 2026-03-27 05:02:11,261 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c36c3f18a3717aa65d3522 2026-03-27 05:02:11,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:02:11,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c36c3f18a3717aa65d3522 HTTP/1.1" 200 None 2026-03-27 05:02:11,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:02:11,292 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-27 05:02:11,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:02:11,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c36c3f18a3717aa65d3522 HTTP/1.1" 200 None 2026-03-27 05:02:11,322 - lava_callback - ERROR - Error processing callback for node 69c36c3f18a3717aa65d3522: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54598 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54060 - "GET / HTTP/1.0" 200 OK 2026-03-27 05:02:41,444 - lava_callback - INFO - Received callback for node 69c36c5e18a3717aa65d3533 INFO: 172.20.0.1:35372 - "POST /node/69c36c5e18a3717aa65d3533 HTTP/1.0" 202 Accepted 2026-03-27 05:02:41,449 - lava_callback - INFO - Starting async job submit for node 69c36c5e18a3717aa65d3533 2026-03-27 05:02:41,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:02:41,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c36c5e18a3717aa65d3533 HTTP/1.1" 200 None 2026-03-27 05:02:41,465 - lava_callback - DEBUG - Processing node 69c36c5e18a3717aa65d3533: baseline-arm-baylibre 2026-03-27 05:02:41,466 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c36c5e18a3717aa65d3533 2026-03-27 05:02:41,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:02:41,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c36c5e18a3717aa65d3533 HTTP/1.1" 200 None 2026-03-27 05:02:41,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:02:41,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-27 05:02:41,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:02:41,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c36c5e18a3717aa65d3533 HTTP/1.1" 200 None 2026-03-27 05:02:41,517 - lava_callback - ERROR - Error processing callback for node 69c36c5e18a3717aa65d3533: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46292 - "GET / HTTP/1.0" 200 OK 2026-03-27 05:03:11,256 - lava_callback - INFO - Received callback for node 69c36c8718a3717aa65d3541 INFO: 172.20.0.1:59128 - "POST /node/69c36c8718a3717aa65d3541 HTTP/1.0" 202 Accepted 2026-03-27 05:03:11,261 - lava_callback - INFO - Starting async job submit for node 69c36c8718a3717aa65d3541 2026-03-27 05:03:11,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:03:11,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c36c8718a3717aa65d3541 HTTP/1.1" 200 None 2026-03-27 05:03:11,277 - lava_callback - DEBUG - Processing node 69c36c8718a3717aa65d3541: baseline-arm-clabbe 2026-03-27 05:03:11,277 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c36c8718a3717aa65d3541 2026-03-27 05:03:11,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:03:11,292 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c36c8718a3717aa65d3541 HTTP/1.1" 200 None 2026-03-27 05:03:11,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:03:11,307 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-27 05:03:11,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:03:11,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c36c8718a3717aa65d3541 HTTP/1.1" 200 None 2026-03-27 05:03:11,345 - lava_callback - ERROR - Error processing callback for node 69c36c8718a3717aa65d3541: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59140 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60330 - "GET / HTTP/1.0" 200 OK 2026-03-27 05:03:51,251 - lava_callback - INFO - Received callback for node 69c36cac18a3717aa65d3550 INFO: 172.20.0.1:34732 - "POST /node/69c36cac18a3717aa65d3550 HTTP/1.0" 202 Accepted 2026-03-27 05:03:51,256 - lava_callback - INFO - Starting async job submit for node 69c36cac18a3717aa65d3550 2026-03-27 05:03:51,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:03:51,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c36cac18a3717aa65d3550 HTTP/1.1" 200 None 2026-03-27 05:03:51,272 - lava_callback - DEBUG - Processing node 69c36cac18a3717aa65d3550: baseline-arm-clabbe 2026-03-27 05:03:51,272 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c36cac18a3717aa65d3550 2026-03-27 05:03:51,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:03:51,287 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c36cac18a3717aa65d3550 HTTP/1.1" 200 None 2026-03-27 05:03:51,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:03:51,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-27 05:03:51,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:03:51,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c36cac18a3717aa65d3550 HTTP/1.1" 200 None 2026-03-27 05:03:51,333 - lava_callback - ERROR - Error processing callback for node 69c36cac18a3717aa65d3550: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45744 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58474 - "GET / HTTP/1.0" 200 OK 2026-03-27 05:04:31,318 - lava_callback - INFO - Received callback for node 69c36cd018a3717aa65d3563 INFO: 172.20.0.1:56402 - "POST /node/69c36cd018a3717aa65d3563 HTTP/1.0" 202 Accepted 2026-03-27 05:04:31,323 - lava_callback - INFO - Starting async job submit for node 69c36cd018a3717aa65d3563 2026-03-27 05:04:31,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:04:31,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c36cd018a3717aa65d3563 HTTP/1.1" 200 None 2026-03-27 05:04:31,338 - lava_callback - DEBUG - Processing node 69c36cd018a3717aa65d3563: baseline-arm-clabbe 2026-03-27 05:04:31,339 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c36cd018a3717aa65d3563 2026-03-27 05:04:31,339 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:04:31,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c36cd018a3717aa65d3563 HTTP/1.1" 200 None 2026-03-27 05:04:31,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:04:31,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-27 05:04:31,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:04:31,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c36cd018a3717aa65d3563 HTTP/1.1" 200 None 2026-03-27 05:04:31,406 - lava_callback - ERROR - Error processing callback for node 69c36cd018a3717aa65d3563: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39824 - "GET / HTTP/1.0" 200 OK 2026-03-27 05:05:01,539 - lava_callback - INFO - Received callback for node 69c36cf218a3717aa65d3570 INFO: 172.20.0.1:33656 - "POST /node/69c36cf218a3717aa65d3570 HTTP/1.0" 202 Accepted 2026-03-27 05:05:01,545 - lava_callback - INFO - Starting async job submit for node 69c36cf218a3717aa65d3570 2026-03-27 05:05:01,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:05:01,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c36cf218a3717aa65d3570 HTTP/1.1" 200 None 2026-03-27 05:05:01,561 - lava_callback - DEBUG - Processing node 69c36cf218a3717aa65d3570: baseline-arm-baylibre 2026-03-27 05:05:01,561 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c36cf218a3717aa65d3570 2026-03-27 05:05:01,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:05:01,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c36cf218a3717aa65d3570 HTTP/1.1" 200 None 2026-03-27 05:05:01,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:05:01,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-27 05:05:01,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:05:01,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c36cf218a3717aa65d3570 HTTP/1.1" 200 None 2026-03-27 05:05:01,641 - lava_callback - ERROR - Error processing callback for node 69c36cf218a3717aa65d3570: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46348 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49088 - "GET / HTTP/1.0" 200 OK 2026-03-27 05:05:41,527 - lava_callback - INFO - Received callback for node 69c36d1618a3717aa65d357d INFO: 172.20.0.1:36542 - "POST /node/69c36d1618a3717aa65d357d HTTP/1.0" 202 Accepted 2026-03-27 05:05:41,533 - lava_callback - INFO - Starting async job submit for node 69c36d1618a3717aa65d357d 2026-03-27 05:05:41,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:05:41,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c36d1618a3717aa65d357d HTTP/1.1" 200 None 2026-03-27 05:05:41,548 - lava_callback - DEBUG - Processing node 69c36d1618a3717aa65d357d: baseline-arm-baylibre 2026-03-27 05:05:41,548 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c36d1618a3717aa65d357d 2026-03-27 05:05:41,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:05:41,563 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c36d1618a3717aa65d357d HTTP/1.1" 200 None 2026-03-27 05:05:41,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:05:41,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-27 05:05:41,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:05:41,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c36d1618a3717aa65d357d HTTP/1.1" 200 None 2026-03-27 05:05:41,606 - lava_callback - ERROR - Error processing callback for node 69c36d1618a3717aa65d357d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40996 - "GET / HTTP/1.0" 200 OK 2026-03-27 05:06:11,417 - lava_callback - INFO - Received callback for node 69c36d3d18a3717aa65d358c INFO: 172.20.0.1:49792 - "POST /node/69c36d3d18a3717aa65d358c HTTP/1.0" 202 Accepted 2026-03-27 05:06:11,422 - lava_callback - INFO - Starting async job submit for node 69c36d3d18a3717aa65d358c 2026-03-27 05:06:11,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:06:11,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c36d3d18a3717aa65d358c HTTP/1.1" 200 None 2026-03-27 05:06:11,444 - lava_callback - DEBUG - Processing node 69c36d3d18a3717aa65d358c: baseline-arm-clabbe 2026-03-27 05:06:11,444 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c36d3d18a3717aa65d358c 2026-03-27 05:06:11,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:06:11,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c36d3d18a3717aa65d358c HTTP/1.1" 200 None 2026-03-27 05:06:11,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:06:11,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-27 05:06:11,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:06:11,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c36d3d18a3717aa65d358c HTTP/1.1" 200 None 2026-03-27 05:06:11,496 - lava_callback - ERROR - Error processing callback for node 69c36d3d18a3717aa65d358c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49800 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55020 - "GET / HTTP/1.0" 200 OK 2026-03-27 05:06:51,365 - lava_callback - INFO - Received callback for node 69c36d6118a3717aa65d359a INFO: 172.20.0.1:48368 - "POST /node/69c36d6118a3717aa65d359a HTTP/1.0" 202 Accepted 2026-03-27 05:06:51,371 - lava_callback - INFO - Starting async job submit for node 69c36d6118a3717aa65d359a 2026-03-27 05:06:51,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:06:51,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c36d6118a3717aa65d359a HTTP/1.1" 200 None 2026-03-27 05:06:51,387 - lava_callback - DEBUG - Processing node 69c36d6118a3717aa65d359a: baseline-arm-clabbe 2026-03-27 05:06:51,387 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c36d6118a3717aa65d359a 2026-03-27 05:06:51,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:06:51,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c36d6118a3717aa65d359a HTTP/1.1" 200 None 2026-03-27 05:06:51,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:06:51,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-27 05:06:51,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:06:51,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c36d6118a3717aa65d359a HTTP/1.1" 200 None 2026-03-27 05:06:51,624 - lava_callback - ERROR - Error processing callback for node 69c36d6118a3717aa65d359a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60136 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53638 - "GET / HTTP/1.0" 200 OK 2026-03-27 05:07:31,470 - lava_callback - INFO - Received callback for node 69c36d8618a3717aa65d35aa INFO: 172.20.0.1:50492 - "POST /node/69c36d8618a3717aa65d35aa HTTP/1.0" 202 Accepted 2026-03-27 05:07:31,475 - lava_callback - INFO - Starting async job submit for node 69c36d8618a3717aa65d35aa 2026-03-27 05:07:31,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:07:31,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c36d8618a3717aa65d35aa HTTP/1.1" 200 None 2026-03-27 05:07:31,492 - lava_callback - DEBUG - Processing node 69c36d8618a3717aa65d35aa: baseline-arm-clabbe 2026-03-27 05:07:31,492 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c36d8618a3717aa65d35aa 2026-03-27 05:07:31,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:07:31,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c36d8618a3717aa65d35aa HTTP/1.1" 200 None 2026-03-27 05:07:31,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:07:31,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-27 05:07:31,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:07:31,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c36d8618a3717aa65d35aa HTTP/1.1" 200 None 2026-03-27 05:07:31,553 - lava_callback - ERROR - Error processing callback for node 69c36d8618a3717aa65d35aa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55378 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42118 - "GET / HTTP/1.0" 200 OK 2026-03-27 05:08:51,533 - lava_callback - INFO - Received callback for node 69c36dce18a3717aa65d35c3 INFO: 172.20.0.1:39764 - "POST /node/69c36dce18a3717aa65d35c3 HTTP/1.0" 202 Accepted 2026-03-27 05:08:51,538 - lava_callback - INFO - Starting async job submit for node 69c36dce18a3717aa65d35c3 2026-03-27 05:08:51,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:08:51,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c36dce18a3717aa65d35c3 HTTP/1.1" 200 None 2026-03-27 05:08:51,554 - lava_callback - DEBUG - Processing node 69c36dce18a3717aa65d35c3: baseline-arm-clabbe 2026-03-27 05:08:51,555 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c36dce18a3717aa65d35c3 2026-03-27 05:08:51,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:08:51,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c36dce18a3717aa65d35c3 HTTP/1.1" 200 None 2026-03-27 05:08:51,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:08:51,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-27 05:08:51,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:08:51,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c36dce18a3717aa65d35c3 HTTP/1.1" 200 None 2026-03-27 05:08:51,631 - lava_callback - ERROR - Error processing callback for node 69c36dce18a3717aa65d35c3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47256 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39284 - "GET / HTTP/1.0" 200 OK 2026-03-27 05:09:51,538 - lava_callback - INFO - Received callback for node 69c36e1418a3717aa65d35d9 INFO: 172.20.0.1:49720 - "POST /node/69c36e1418a3717aa65d35d9 HTTP/1.0" 202 Accepted 2026-03-27 05:09:51,544 - lava_callback - INFO - Starting async job submit for node 69c36e1418a3717aa65d35d9 2026-03-27 05:09:51,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:09:51,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c36e1418a3717aa65d35d9 HTTP/1.1" 200 None 2026-03-27 05:09:51,595 - lava_callback - DEBUG - Processing node 69c36e1418a3717aa65d35d9: baseline-arm-clabbe 2026-03-27 05:09:51,595 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c36e1418a3717aa65d35d9 2026-03-27 05:09:51,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:09:51,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c36e1418a3717aa65d35d9 HTTP/1.1" 200 None 2026-03-27 05:09:51,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:09:51,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-27 05:09:51,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:09:51,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c36e1418a3717aa65d35d9 HTTP/1.1" 200 None 2026-03-27 05:09:51,660 - lava_callback - ERROR - Error processing callback for node 69c36e1418a3717aa65d35d9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46326 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43674 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43972 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45864 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33598 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39676 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35672 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53722 - "GET / HTTP/1.0" 200 OK 2026-03-27 05:16:51,795 - lava_callback - INFO - Received callback for node 69c36fbd18a3717aa65d3620 INFO: 172.20.0.1:37472 - "POST /node/69c36fbd18a3717aa65d3620 HTTP/1.0" 202 Accepted 2026-03-27 05:16:51,800 - lava_callback - INFO - Starting async job submit for node 69c36fbd18a3717aa65d3620 2026-03-27 05:16:51,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:16:51,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c36fbd18a3717aa65d3620 HTTP/1.1" 200 None 2026-03-27 05:16:51,824 - lava_callback - DEBUG - Processing node 69c36fbd18a3717aa65d3620: baseline-arm-clabbe 2026-03-27 05:16:51,824 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c36fbd18a3717aa65d3620 2026-03-27 05:16:51,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:16:51,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c36fbd18a3717aa65d3620 HTTP/1.1" 200 None 2026-03-27 05:16:51,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:16:51,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-27 05:16:51,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:16:51,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c36fbd18a3717aa65d3620 HTTP/1.1" 200 None 2026-03-27 05:16:51,883 - lava_callback - ERROR - Error processing callback for node 69c36fbd18a3717aa65d3620: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34050 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34496 - "GET / HTTP/1.0" 200 OK 2026-03-27 05:17:42,241 - lava_callback - INFO - Received callback for node 69c36fe618a3717aa65d3634 INFO: 172.20.0.1:33250 - "POST /node/69c36fe618a3717aa65d3634 HTTP/1.0" 202 Accepted 2026-03-27 05:17:42,246 - lava_callback - INFO - Starting async job submit for node 69c36fe618a3717aa65d3634 2026-03-27 05:17:42,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:17:42,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c36fe618a3717aa65d3634 HTTP/1.1" 200 None 2026-03-27 05:17:42,262 - lava_callback - DEBUG - Processing node 69c36fe618a3717aa65d3634: baseline-arm64-baylibre 2026-03-27 05:17:42,263 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c36fe618a3717aa65d3634 2026-03-27 05:17:42,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:17:42,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c36fe618a3717aa65d3634 HTTP/1.1" 200 None 2026-03-27 05:17:42,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:17:42,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-27 05:17:42,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:17:42,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c36fe618a3717aa65d3634 HTTP/1.1" 200 None 2026-03-27 05:17:42,326 - lava_callback - ERROR - Error processing callback for node 69c36fe618a3717aa65d3634: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39540 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48282 - "GET / HTTP/1.0" 200 OK 2026-03-27 05:19:02,212 - lava_callback - INFO - Received callback for node 69c3703318a3717aa65d363b INFO: 172.20.0.1:57368 - "POST /node/69c3703318a3717aa65d363b HTTP/1.0" 202 Accepted 2026-03-27 05:19:02,218 - lava_callback - INFO - Starting async job submit for node 69c3703318a3717aa65d363b 2026-03-27 05:19:02,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:19:02,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3703318a3717aa65d363b HTTP/1.1" 200 None 2026-03-27 05:19:02,234 - lava_callback - DEBUG - Processing node 69c3703318a3717aa65d363b: baseline-arm64-baylibre 2026-03-27 05:19:02,234 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3703318a3717aa65d363b 2026-03-27 05:19:02,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:19:02,247 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3703318a3717aa65d363b HTTP/1.1" 200 None 2026-03-27 05:19:02,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:19:02,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-27 05:19:02,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:19:02,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3703318a3717aa65d363b HTTP/1.1" 200 None 2026-03-27 05:19:02,294 - lava_callback - ERROR - Error processing callback for node 69c3703318a3717aa65d363b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60332 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33220 - "GET / HTTP/1.0" 200 OK 2026-03-27 05:20:02,288 - lava_callback - INFO - Received callback for node 69c3708018a3717aa65d3649 INFO: 172.20.0.1:41852 - "POST /node/69c3708018a3717aa65d3649 HTTP/1.0" 202 Accepted 2026-03-27 05:20:02,294 - lava_callback - INFO - Starting async job submit for node 69c3708018a3717aa65d3649 2026-03-27 05:20:02,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:20:02,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3708018a3717aa65d3649 HTTP/1.1" 200 None 2026-03-27 05:20:02,310 - lava_callback - DEBUG - Processing node 69c3708018a3717aa65d3649: baseline-arm64-baylibre 2026-03-27 05:20:02,310 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3708018a3717aa65d3649 2026-03-27 05:20:02,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:20:02,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3708018a3717aa65d3649 HTTP/1.1" 200 None 2026-03-27 05:20:02,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:20:02,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-27 05:20:02,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:20:02,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3708018a3717aa65d3649 HTTP/1.1" 200 None 2026-03-27 05:20:02,368 - lava_callback - ERROR - Error processing callback for node 69c3708018a3717aa65d3649: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-27 05:20:03,644 - lava_callback - INFO - Received callback for node 69c5c125abf66a8f4d9bd11b INFO: 172.20.0.1:41860 - "POST /node/69c5c125abf66a8f4d9bd11b HTTP/1.0" 202 Accepted 2026-03-27 05:20:03,650 - lava_callback - INFO - Starting async job submit for node 69c5c125abf66a8f4d9bd11b 2026-03-27 05:20:03,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:20:03,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5c125abf66a8f4d9bd11b HTTP/1.1" 200 None 2026-03-27 05:20:03,714 - lava_callback - DEBUG - Processing node 69c5c125abf66a8f4d9bd11b: baseline-nfs-arm64 2026-03-27 05:20:04,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 05:20:04,392 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 05:20:04,394 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5c125abf66a8f4d9bd11b/log.txt.gz 2026-03-27 05:20:04,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 05:20:04,933 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 05:20:04,935 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5c125abf66a8f4d9bd11b/lava-logs.yaml 2026-03-27 05:20:04,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 05:20:05,163 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 05:20:05,165 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5c125abf66a8f4d9bd11b/lava_callback.json.gz 2026-03-27 05:20:05,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:20:05,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5c125abf66a8f4d9bd11b HTTP/1.1" 200 None 2026-03-27 05:20:05,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:20:05,221 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-27 05:20:05,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:20:05,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5c125abf66a8f4d9bd11b HTTP/1.1" 200 None 2026-03-27 05:20:05,298 - lava_callback - INFO - Completed processing callback for node 69c5c125abf66a8f4d9bd11b INFO: 172.20.0.1:41864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46234 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38710 - "GET / HTTP/1.0" 200 OK 2026-03-27 05:20:31,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:20:31,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:41702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36904 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36922 - "GET / HTTP/1.0" 200 OK 2026-03-27 05:21:22,380 - lava_callback - INFO - Received callback for node 69c370cf18a3717aa65d364f INFO: 172.20.0.1:45832 - "POST /node/69c370cf18a3717aa65d364f HTTP/1.0" 202 Accepted 2026-03-27 05:21:22,386 - lava_callback - INFO - Starting async job submit for node 69c370cf18a3717aa65d364f 2026-03-27 05:21:22,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:21:22,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c370cf18a3717aa65d364f HTTP/1.1" 200 None 2026-03-27 05:21:22,408 - lava_callback - DEBUG - Processing node 69c370cf18a3717aa65d364f: baseline-arm64-baylibre 2026-03-27 05:21:22,408 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c370cf18a3717aa65d364f 2026-03-27 05:21:22,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:21:22,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c370cf18a3717aa65d364f HTTP/1.1" 200 None 2026-03-27 05:21:22,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:21:22,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-27 05:21:22,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:21:22,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c370cf18a3717aa65d364f HTTP/1.1" 200 None 2026-03-27 05:21:22,475 - lava_callback - ERROR - Error processing callback for node 69c370cf18a3717aa65d364f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49972 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36002 - "GET / HTTP/1.0" 200 OK 2026-03-27 05:22:42,470 - lava_callback - INFO - Received callback for node 69c3711d18a3717aa65d365e INFO: 172.20.0.1:59578 - "POST /node/69c3711d18a3717aa65d365e HTTP/1.0" 202 Accepted 2026-03-27 05:22:42,476 - lava_callback - INFO - Starting async job submit for node 69c3711d18a3717aa65d365e 2026-03-27 05:22:42,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:22:42,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3711d18a3717aa65d365e HTTP/1.1" 200 None 2026-03-27 05:22:42,493 - lava_callback - DEBUG - Processing node 69c3711d18a3717aa65d365e: baseline-arm64-baylibre 2026-03-27 05:22:42,493 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3711d18a3717aa65d365e 2026-03-27 05:22:42,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:22:42,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3711d18a3717aa65d365e HTTP/1.1" 200 None 2026-03-27 05:22:42,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:22:42,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-27 05:22:42,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:22:42,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3711d18a3717aa65d365e HTTP/1.1" 200 None 2026-03-27 05:22:42,544 - lava_callback - ERROR - Error processing callback for node 69c3711d18a3717aa65d365e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46682 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33230 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55330 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59124 - "GET / HTTP/1.0" 200 OK 2026-03-27 05:25:32,213 - lava_callback - INFO - Received callback for node 69c371bf18a3717aa65d367c INFO: 172.20.0.1:33822 - "POST /node/69c371bf18a3717aa65d367c HTTP/1.0" 202 Accepted 2026-03-27 05:25:32,219 - lava_callback - INFO - Starting async job submit for node 69c371bf18a3717aa65d367c 2026-03-27 05:25:32,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:25:32,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c371bf18a3717aa65d367c HTTP/1.1" 200 None 2026-03-27 05:25:32,235 - lava_callback - DEBUG - Processing node 69c371bf18a3717aa65d367c: baseline-arm64-clabbe 2026-03-27 05:25:32,235 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c371bf18a3717aa65d367c 2026-03-27 05:25:32,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:25:32,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c371bf18a3717aa65d367c HTTP/1.1" 200 None 2026-03-27 05:25:32,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:25:32,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-27 05:25:32,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:25:32,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c371bf18a3717aa65d367c HTTP/1.1" 200 None 2026-03-27 05:25:32,301 - lava_callback - ERROR - Error processing callback for node 69c371bf18a3717aa65d367c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60386 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46316 - "GET / HTTP/1.0" 200 OK 2026-03-27 05:26:32,208 - lava_callback - INFO - Received callback for node 69c3720018a3717aa65d368f INFO: 172.20.0.1:48770 - "POST /node/69c3720018a3717aa65d368f HTTP/1.0" 202 Accepted 2026-03-27 05:26:32,214 - lava_callback - INFO - Starting async job submit for node 69c3720018a3717aa65d368f 2026-03-27 05:26:32,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:26:32,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3720018a3717aa65d368f HTTP/1.1" 200 None 2026-03-27 05:26:32,235 - lava_callback - DEBUG - Processing node 69c3720018a3717aa65d368f: baseline-arm-clabbe 2026-03-27 05:26:32,235 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3720018a3717aa65d368f 2026-03-27 05:26:32,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:26:32,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3720018a3717aa65d368f HTTP/1.1" 200 None 2026-03-27 05:26:32,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:26:32,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-27 05:26:32,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:26:32,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3720018a3717aa65d368f HTTP/1.1" 200 None 2026-03-27 05:26:32,313 - lava_callback - ERROR - Error processing callback for node 69c3720018a3717aa65d368f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45074 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42880 - "GET / HTTP/1.0" 200 OK 2026-03-27 05:27:32,251 - lava_callback - INFO - Received callback for node 69c3723218a3717aa65d3698 INFO: 172.20.0.1:60200 - "POST /node/69c3723218a3717aa65d3698 HTTP/1.0" 202 Accepted 2026-03-27 05:27:32,256 - lava_callback - INFO - Starting async job submit for node 69c3723218a3717aa65d3698 2026-03-27 05:27:32,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:27:32,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3723218a3717aa65d3698 HTTP/1.1" 200 None 2026-03-27 05:27:32,272 - lava_callback - DEBUG - Processing node 69c3723218a3717aa65d3698: baseline-arm64-clabbe 2026-03-27 05:27:32,273 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3723218a3717aa65d3698 2026-03-27 05:27:32,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:27:32,287 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3723218a3717aa65d3698 HTTP/1.1" 200 None 2026-03-27 05:27:32,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:27:32,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-27 05:27:32,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:27:32,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3723218a3717aa65d3698 HTTP/1.1" 200 None 2026-03-27 05:27:32,331 - lava_callback - ERROR - Error processing callback for node 69c3723218a3717aa65d3698: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35320 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38732 - "GET / HTTP/1.0" 200 OK 2026-03-27 05:28:34,601 - lava_callback - INFO - Received callback for node 69c5b897abf66a8f4d9bccef INFO: 172.20.0.1:53392 - "POST /node/69c5b897abf66a8f4d9bccef HTTP/1.0" 202 Accepted 2026-03-27 05:28:34,607 - lava_callback - INFO - Starting async job submit for node 69c5b897abf66a8f4d9bccef 2026-03-27 05:28:34,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:28:34,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b897abf66a8f4d9bccef HTTP/1.1" 200 None 2026-03-27 05:28:34,645 - lava_callback - DEBUG - Processing node 69c5b897abf66a8f4d9bccef: baseline-arm64-broonie 2026-03-27 05:28:34,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 05:28:35,211 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 05:28:35,213 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c5b897abf66a8f4d9bccef/log.txt.gz 2026-03-27 05:28:35,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 05:28:35,762 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 05:28:35,764 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c5b897abf66a8f4d9bccef/lava-logs.yaml 2026-03-27 05:28:35,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 05:28:35,932 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 05:28:35,934 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c5b897abf66a8f4d9bccef/lava_callback.json.gz 2026-03-27 05:28:35,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:28:35,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b897abf66a8f4d9bccef HTTP/1.1" 200 None 2026-03-27 05:28:35,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:28:35,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-27 05:28:35,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:28:36,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5b897abf66a8f4d9bccef HTTP/1.1" 200 None 2026-03-27 05:28:36,051 - lava_callback - INFO - Completed processing callback for node 69c5b897abf66a8f4d9bccef INFO: 172.20.0.1:53394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42488 - "GET / HTTP/1.0" 200 OK 2026-03-27 05:28:52,336 - lava_callback - INFO - Received callback for node 69c3728018a3717aa65d36a7 INFO: 172.20.0.1:56212 - "POST /node/69c3728018a3717aa65d36a7 HTTP/1.0" 202 Accepted 2026-03-27 05:28:52,341 - lava_callback - INFO - Starting async job submit for node 69c3728018a3717aa65d36a7 2026-03-27 05:28:52,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:28:52,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3728018a3717aa65d36a7 HTTP/1.1" 200 None 2026-03-27 05:28:52,361 - lava_callback - DEBUG - Processing node 69c3728018a3717aa65d36a7: baseline-arm64-clabbe 2026-03-27 05:28:52,361 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3728018a3717aa65d36a7 2026-03-27 05:28:52,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:28:52,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3728018a3717aa65d36a7 HTTP/1.1" 200 None 2026-03-27 05:28:52,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:28:52,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-27 05:28:52,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:28:52,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3728018a3717aa65d36a7 HTTP/1.1" 200 None 2026-03-27 05:28:52,424 - lava_callback - ERROR - Error processing callback for node 69c3728018a3717aa65d36a7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56218 - "GET / HTTP/1.0" 200 OK 2026-03-27 05:29:01,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:29:01,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:44714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41834 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45426 - "GET / HTTP/1.0" 200 OK 2026-03-27 05:30:12,384 - lava_callback - INFO - Received callback for node 69c372ce18a3717aa65d36b2 INFO: 172.20.0.1:46998 - "POST /node/69c372ce18a3717aa65d36b2 HTTP/1.0" 202 Accepted 2026-03-27 05:30:12,390 - lava_callback - INFO - Starting async job submit for node 69c372ce18a3717aa65d36b2 2026-03-27 05:30:12,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:30:12,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c372ce18a3717aa65d36b2 HTTP/1.1" 200 None 2026-03-27 05:30:12,413 - lava_callback - DEBUG - Processing node 69c372ce18a3717aa65d36b2: baseline-arm64-clabbe 2026-03-27 05:30:12,413 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c372ce18a3717aa65d36b2 2026-03-27 05:30:12,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:30:12,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c372ce18a3717aa65d36b2 HTTP/1.1" 200 None 2026-03-27 05:30:12,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:30:12,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-27 05:30:12,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:30:12,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c372ce18a3717aa65d36b2 HTTP/1.1" 200 None 2026-03-27 05:30:12,499 - lava_callback - ERROR - Error processing callback for node 69c372ce18a3717aa65d36b2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47010 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42000 - "GET / HTTP/1.0" 200 OK 2026-03-27 05:30:35,188 - lava_callback - INFO - Received callback for node 69c5d5a32cb62322ad966327 INFO: 172.20.0.1:57524 - "POST /node/69c5d5a32cb62322ad966327 HTTP/1.0" 202 Accepted 2026-03-27 05:30:35,194 - lava_callback - INFO - Starting async job submit for node 69c5d5a32cb62322ad966327 2026-03-27 05:30:35,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:30:35,231 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5a32cb62322ad966327 HTTP/1.1" 200 None 2026-03-27 05:30:35,232 - lava_callback - DEBUG - Processing node 69c5d5a32cb62322ad966327: baseline-arm64-broonie 2026-03-27 05:30:35,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 05:30:35,673 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 05:30:35,674 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c5d5a32cb62322ad966327/log.txt.gz 2026-03-27 05:30:35,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 05:30:36,085 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 05:30:36,087 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c5d5a32cb62322ad966327/lava-logs.yaml 2026-03-27 05:30:36,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 05:30:36,274 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 05:30:36,276 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c5d5a32cb62322ad966327/lava_callback.json.gz 2026-03-27 05:30:36,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:30:36,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5a32cb62322ad966327 HTTP/1.1" 200 None 2026-03-27 05:30:36,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:30:36,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-27 05:30:36,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:30:36,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d5a32cb62322ad966327 HTTP/1.1" 200 None 2026-03-27 05:30:36,382 - lava_callback - INFO - Completed processing callback for node 69c5d5a32cb62322ad966327 INFO: 172.20.0.1:57536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56812 - "GET / HTTP/1.0" 200 OK 2026-03-27 05:31:01,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:31:01,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:47532 - "GET / HTTP/1.0" 200 OK 2026-03-27 05:31:12,475 - lava_callback - INFO - Received callback for node 69c3731d18a3717aa65d36c3 INFO: 172.20.0.1:49770 - "POST /node/69c3731d18a3717aa65d36c3 HTTP/1.0" 202 Accepted 2026-03-27 05:31:12,481 - lava_callback - INFO - Starting async job submit for node 69c3731d18a3717aa65d36c3 2026-03-27 05:31:12,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:31:12,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3731d18a3717aa65d36c3 HTTP/1.1" 200 None 2026-03-27 05:31:12,506 - lava_callback - DEBUG - Processing node 69c3731d18a3717aa65d36c3: baseline-arm64-clabbe 2026-03-27 05:31:12,507 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3731d18a3717aa65d36c3 2026-03-27 05:31:12,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:31:12,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3731d18a3717aa65d36c3 HTTP/1.1" 200 None 2026-03-27 05:31:12,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:31:12,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-27 05:31:12,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:31:12,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3731d18a3717aa65d36c3 HTTP/1.1" 200 None 2026-03-27 05:31:12,558 - lava_callback - ERROR - Error processing callback for node 69c3731d18a3717aa65d36c3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49774 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44378 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60944 - "GET / HTTP/1.0" 200 OK 2026-03-27 05:32:32,520 - lava_callback - INFO - Received callback for node 69c3736b18a3717aa65d36d3 INFO: 172.20.0.1:41134 - "POST /node/69c3736b18a3717aa65d36d3 HTTP/1.0" 202 Accepted 2026-03-27 05:32:32,525 - lava_callback - INFO - Starting async job submit for node 69c3736b18a3717aa65d36d3 2026-03-27 05:32:32,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:32:32,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3736b18a3717aa65d36d3 HTTP/1.1" 200 None 2026-03-27 05:32:32,552 - lava_callback - DEBUG - Processing node 69c3736b18a3717aa65d36d3: baseline-arm64-clabbe 2026-03-27 05:32:32,552 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3736b18a3717aa65d36d3 2026-03-27 05:32:32,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:32:32,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3736b18a3717aa65d36d3 HTTP/1.1" 200 None 2026-03-27 05:32:32,567 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:32:32,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-27 05:32:32,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:32:32,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3736b18a3717aa65d36d3 HTTP/1.1" 200 None 2026-03-27 05:32:32,606 - lava_callback - ERROR - Error processing callback for node 69c3736b18a3717aa65d36d3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-27 05:32:34,535 - lava_callback - INFO - Received callback for node 69c611ce2cb62322ad969541 INFO: 172.20.0.1:41142 - "POST /node/69c611ce2cb62322ad969541 HTTP/1.0" 202 Accepted 2026-03-27 05:32:34,541 - lava_callback - INFO - Starting async job submit for node 69c611ce2cb62322ad969541 2026-03-27 05:32:34,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:32:34,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c611ce2cb62322ad969541 HTTP/1.1" 200 None 2026-03-27 05:32:34,580 - lava_callback - DEBUG - Processing node 69c611ce2cb62322ad969541: baseline-arm64-broonie 2026-03-27 05:32:34,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 05:32:34,993 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 05:32:34,995 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c611ce2cb62322ad969541/log.txt.gz 2026-03-27 05:32:35,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 05:32:35,388 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 05:32:35,390 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c611ce2cb62322ad969541/lava-logs.yaml 2026-03-27 05:32:35,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 05:32:35,572 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 05:32:35,574 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c611ce2cb62322ad969541/lava_callback.json.gz 2026-03-27 05:32:35,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:32:35,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c611ce2cb62322ad969541 HTTP/1.1" 200 None 2026-03-27 05:32:35,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:32:35,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-27 05:32:35,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:32:35,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c611ce2cb62322ad969541 HTTP/1.1" 200 None 2026-03-27 05:32:35,665 - lava_callback - INFO - Completed processing callback for node 69c611ce2cb62322ad969541 INFO: 172.20.0.1:41154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51828 - "GET / HTTP/1.0" 200 OK 2026-03-27 05:33:01,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:33:01,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:50196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42414 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42650 - "GET / HTTP/1.0" 200 OK 2026-03-27 05:33:52,499 - lava_callback - INFO - Received callback for node 69c373aa18a3717aa65d36e1 INFO: 172.20.0.1:59434 - "POST /node/69c373aa18a3717aa65d36e1 HTTP/1.0" 202 Accepted 2026-03-27 05:33:52,504 - lava_callback - INFO - Starting async job submit for node 69c373aa18a3717aa65d36e1 2026-03-27 05:33:52,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:33:52,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c373aa18a3717aa65d36e1 HTTP/1.1" 200 None 2026-03-27 05:33:52,521 - lava_callback - DEBUG - Processing node 69c373aa18a3717aa65d36e1: baseline-arm-clabbe 2026-03-27 05:33:52,521 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c373aa18a3717aa65d36e1 2026-03-27 05:33:52,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:33:52,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c373aa18a3717aa65d36e1 HTTP/1.1" 200 None 2026-03-27 05:33:52,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:33:52,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-27 05:33:52,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:33:52,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c373aa18a3717aa65d36e1 HTTP/1.1" 200 None 2026-03-27 05:33:52,580 - lava_callback - ERROR - Error processing callback for node 69c373aa18a3717aa65d36e1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43508 - "GET / HTTP/1.0" 200 OK 2026-03-27 05:34:12,488 - lava_callback - INFO - Received callback for node 69c373cf18a3717aa65d36ef INFO: 172.20.0.1:38170 - "POST /node/69c373cf18a3717aa65d36ef HTTP/1.0" 202 Accepted 2026-03-27 05:34:12,493 - lava_callback - INFO - Starting async job submit for node 69c373cf18a3717aa65d36ef 2026-03-27 05:34:12,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:34:12,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c373cf18a3717aa65d36ef HTTP/1.1" 200 None 2026-03-27 05:34:12,510 - lava_callback - DEBUG - Processing node 69c373cf18a3717aa65d36ef: baseline-arm-clabbe 2026-03-27 05:34:12,510 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c373cf18a3717aa65d36ef 2026-03-27 05:34:12,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:34:12,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c373cf18a3717aa65d36ef HTTP/1.1" 200 None 2026-03-27 05:34:12,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:34:12,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-27 05:34:12,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:34:12,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c373cf18a3717aa65d36ef HTTP/1.1" 200 None 2026-03-27 05:34:12,586 - lava_callback - ERROR - Error processing callback for node 69c373cf18a3717aa65d36ef: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38182 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49592 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44320 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51652 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45148 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37170 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38916 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44056 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55904 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35352 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48442 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45130 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45620 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59398 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48986 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49016 - "GET / HTTP/1.0" 200 OK 2026-03-27 05:48:23,959 - lava_callback - INFO - Received callback for node 69c3772518a3717aa65d3727 INFO: 172.20.0.1:53390 - "POST /node/69c3772518a3717aa65d3727 HTTP/1.0" 202 Accepted 2026-03-27 05:48:23,965 - lava_callback - INFO - Starting async job submit for node 69c3772518a3717aa65d3727 2026-03-27 05:48:23,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:48:23,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3772518a3717aa65d3727 HTTP/1.1" 200 None 2026-03-27 05:48:23,980 - lava_callback - DEBUG - Processing node 69c3772518a3717aa65d3727: baseline-arm64-baylibre 2026-03-27 05:48:23,980 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3772518a3717aa65d3727 2026-03-27 05:48:23,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:48:24,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3772518a3717aa65d3727 HTTP/1.1" 200 None 2026-03-27 05:48:24,015 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:48:24,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-27 05:48:24,030 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:48:24,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3772518a3717aa65d3727 HTTP/1.1" 200 None 2026-03-27 05:48:24,052 - lava_callback - ERROR - Error processing callback for node 69c3772518a3717aa65d3727: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34844 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55120 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54598 - "GET / HTTP/1.0" 200 OK 2026-03-27 05:50:43,992 - lava_callback - INFO - Received callback for node 69c377b118a3717aa65d3738 INFO: 172.20.0.1:42412 - "POST /node/69c377b118a3717aa65d3738 HTTP/1.0" 202 Accepted 2026-03-27 05:50:43,997 - lava_callback - INFO - Starting async job submit for node 69c377b118a3717aa65d3738 2026-03-27 05:50:43,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:50:44,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c377b118a3717aa65d3738 HTTP/1.1" 200 None 2026-03-27 05:50:44,013 - lava_callback - DEBUG - Processing node 69c377b118a3717aa65d3738: baseline-arm64-baylibre 2026-03-27 05:50:44,014 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c377b118a3717aa65d3738 2026-03-27 05:50:44,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:50:44,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c377b118a3717aa65d3738 HTTP/1.1" 200 None 2026-03-27 05:50:44,029 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:50:44,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-27 05:50:44,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:50:44,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c377b118a3717aa65d3738 HTTP/1.1" 200 None 2026-03-27 05:50:44,068 - lava_callback - ERROR - Error processing callback for node 69c377b118a3717aa65d3738: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40720 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40120 - "GET / HTTP/1.0" 200 OK 2026-03-27 05:52:04,036 - lava_callback - INFO - Received callback for node 69c3780118a3717aa65d3746 INFO: 172.20.0.1:48704 - "POST /node/69c3780118a3717aa65d3746 HTTP/1.0" 202 Accepted 2026-03-27 05:52:04,041 - lava_callback - INFO - Starting async job submit for node 69c3780118a3717aa65d3746 2026-03-27 05:52:04,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:52:04,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3780118a3717aa65d3746 HTTP/1.1" 200 None 2026-03-27 05:52:04,057 - lava_callback - DEBUG - Processing node 69c3780118a3717aa65d3746: baseline-arm64-baylibre 2026-03-27 05:52:04,057 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3780118a3717aa65d3746 2026-03-27 05:52:04,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:52:04,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3780118a3717aa65d3746 HTTP/1.1" 200 None 2026-03-27 05:52:04,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:52:04,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-27 05:52:04,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:52:04,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3780118a3717aa65d3746 HTTP/1.1" 200 None 2026-03-27 05:52:04,131 - lava_callback - ERROR - Error processing callback for node 69c3780118a3717aa65d3746: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50650 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48106 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48124 - "GET / HTTP/1.0" 200 OK 2026-03-27 05:53:24,137 - lava_callback - INFO - Received callback for node 69c3785218a3717aa65d3750 INFO: 172.20.0.1:39904 - "POST /node/69c3785218a3717aa65d3750 HTTP/1.0" 202 Accepted 2026-03-27 05:53:24,142 - lava_callback - INFO - Starting async job submit for node 69c3785218a3717aa65d3750 2026-03-27 05:53:24,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:53:24,157 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3785218a3717aa65d3750 HTTP/1.1" 200 None 2026-03-27 05:53:24,158 - lava_callback - DEBUG - Processing node 69c3785218a3717aa65d3750: baseline-arm64-baylibre 2026-03-27 05:53:24,158 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3785218a3717aa65d3750 2026-03-27 05:53:24,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:53:24,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3785218a3717aa65d3750 HTTP/1.1" 200 None 2026-03-27 05:53:24,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:53:24,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-27 05:53:24,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:53:24,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3785218a3717aa65d3750 HTTP/1.1" 200 None 2026-03-27 05:53:24,210 - lava_callback - ERROR - Error processing callback for node 69c3785218a3717aa65d3750: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53372 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40512 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38034 - "GET / HTTP/1.0" 200 OK 2026-03-27 05:55:44,251 - lava_callback - INFO - Received callback for node 69c378cc18a3717aa65d3763 INFO: 172.20.0.1:54106 - "POST /node/69c378cc18a3717aa65d3763 HTTP/1.0" 202 Accepted 2026-03-27 05:55:44,256 - lava_callback - INFO - Starting async job submit for node 69c378cc18a3717aa65d3763 2026-03-27 05:55:44,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:55:44,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c378cc18a3717aa65d3763 HTTP/1.1" 200 None 2026-03-27 05:55:44,277 - lava_callback - DEBUG - Processing node 69c378cc18a3717aa65d3763: baseline-arm64-baylibre 2026-03-27 05:55:44,277 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c378cc18a3717aa65d3763 2026-03-27 05:55:44,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:55:44,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c378cc18a3717aa65d3763 HTTP/1.1" 200 None 2026-03-27 05:55:44,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:55:44,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-27 05:55:44,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:55:44,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c378cc18a3717aa65d3763 HTTP/1.1" 200 None 2026-03-27 05:55:44,339 - lava_callback - ERROR - Error processing callback for node 69c378cc18a3717aa65d3763: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48962 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53164 - "GET / HTTP/1.0" 200 OK 2026-03-27 05:57:04,300 - lava_callback - INFO - Received callback for node 69c3791b18a3717aa65d376c INFO: 172.20.0.1:43774 - "POST /node/69c3791b18a3717aa65d376c HTTP/1.0" 202 Accepted 2026-03-27 05:57:04,305 - lava_callback - INFO - Starting async job submit for node 69c3791b18a3717aa65d376c 2026-03-27 05:57:04,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:57:04,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3791b18a3717aa65d376c HTTP/1.1" 200 None 2026-03-27 05:57:04,330 - lava_callback - DEBUG - Processing node 69c3791b18a3717aa65d376c: baseline-arm64-baylibre 2026-03-27 05:57:04,330 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3791b18a3717aa65d376c 2026-03-27 05:57:04,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:57:04,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3791b18a3717aa65d376c HTTP/1.1" 200 None 2026-03-27 05:57:04,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:57:04,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-27 05:57:04,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:57:04,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3791b18a3717aa65d376c HTTP/1.1" 200 None 2026-03-27 05:57:04,384 - lava_callback - ERROR - Error processing callback for node 69c3791b18a3717aa65d376c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41506 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54798 - "GET / HTTP/1.0" 200 OK 2026-03-27 05:58:12,853 - lava_callback - INFO - Received callback for node 69c5c8e3abf66a8f4d9bd31d INFO: 172.20.0.1:60714 - "POST /node/69c5c8e3abf66a8f4d9bd31d HTTP/1.0" 202 Accepted 2026-03-27 05:58:12,862 - lava_callback - INFO - Starting async job submit for node 69c5c8e3abf66a8f4d9bd31d 2026-03-27 05:58:12,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:58:12,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5c8e3abf66a8f4d9bd31d HTTP/1.1" 200 None 2026-03-27 05:58:12,908 - lava_callback - DEBUG - Processing node 69c5c8e3abf66a8f4d9bd31d: baseline-arm64 2026-03-27 05:58:13,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 05:58:13,583 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 05:58:13,584 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5c8e3abf66a8f4d9bd31d/log.txt.gz 2026-03-27 05:58:13,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 05:58:14,070 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 05:58:14,072 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5c8e3abf66a8f4d9bd31d/lava-logs.yaml 2026-03-27 05:58:14,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 05:58:14,268 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 05:58:14,270 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5c8e3abf66a8f4d9bd31d/lava_callback.json.gz 2026-03-27 05:58:14,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:58:14,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5c8e3abf66a8f4d9bd31d HTTP/1.1" 200 None 2026-03-27 05:58:14,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:58:14,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-27 05:58:14,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:58:14,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5c8e3abf66a8f4d9bd31d HTTP/1.1" 200 None 2026-03-27 05:58:14,436 - lava_callback - INFO - Completed processing callback for node 69c5c8e3abf66a8f4d9bd31d INFO: 172.20.0.1:60722 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48804 - "GET / HTTP/1.0" 200 OK 2026-03-27 05:58:31,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:58:31,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:42540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55268 - "GET / HTTP/1.0" 200 OK 2026-03-27 05:59:13,591 - lava_callback - INFO - Received callback for node 69c3799b18a3717aa65d377c INFO: 172.20.0.1:56562 - "POST /node/69c3799b18a3717aa65d377c HTTP/1.0" 202 Accepted 2026-03-27 05:59:13,597 - lava_callback - INFO - Starting async job submit for node 69c3799b18a3717aa65d377c 2026-03-27 05:59:13,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:59:13,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3799b18a3717aa65d377c HTTP/1.1" 200 None 2026-03-27 05:59:13,613 - lava_callback - DEBUG - Processing node 69c3799b18a3717aa65d377c: baseline-arm64-clabbe 2026-03-27 05:59:13,613 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3799b18a3717aa65d377c 2026-03-27 05:59:13,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:59:13,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3799b18a3717aa65d377c HTTP/1.1" 200 None 2026-03-27 05:59:13,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:59:13,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-27 05:59:13,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 05:59:13,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3799b18a3717aa65d377c HTTP/1.1" 200 None 2026-03-27 05:59:13,669 - lava_callback - ERROR - Error processing callback for node 69c3799b18a3717aa65d377c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56570 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45898 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49010 - "GET / HTTP/1.0" 200 OK 2026-03-27 06:00:53,636 - lava_callback - INFO - Received callback for node 69c37a0018a3717aa65d378d INFO: 172.20.0.1:55844 - "POST /node/69c37a0018a3717aa65d378d HTTP/1.0" 202 Accepted 2026-03-27 06:00:53,642 - lava_callback - INFO - Starting async job submit for node 69c37a0018a3717aa65d378d 2026-03-27 06:00:53,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:00:53,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c37a0018a3717aa65d378d HTTP/1.1" 200 None 2026-03-27 06:00:53,659 - lava_callback - DEBUG - Processing node 69c37a0018a3717aa65d378d: baseline-arm64-clabbe 2026-03-27 06:00:53,659 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c37a0018a3717aa65d378d 2026-03-27 06:00:53,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:00:53,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c37a0018a3717aa65d378d HTTP/1.1" 200 None 2026-03-27 06:00:53,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:00:53,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-27 06:00:53,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:00:53,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c37a0018a3717aa65d378d HTTP/1.1" 200 None 2026-03-27 06:00:53,724 - lava_callback - ERROR - Error processing callback for node 69c37a0018a3717aa65d378d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37844 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33258 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39920 - "GET / HTTP/1.0" 200 OK 2026-03-27 06:03:13,710 - lava_callback - INFO - Received callback for node 69c37a9918a3717aa65d37a4 INFO: 172.20.0.1:38152 - "POST /node/69c37a9918a3717aa65d37a4 HTTP/1.0" 202 Accepted 2026-03-27 06:03:13,715 - lava_callback - INFO - Starting async job submit for node 69c37a9918a3717aa65d37a4 2026-03-27 06:03:13,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:03:13,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c37a9918a3717aa65d37a4 HTTP/1.1" 200 None 2026-03-27 06:03:13,754 - lava_callback - DEBUG - Processing node 69c37a9918a3717aa65d37a4: baseline-arm64-clabbe 2026-03-27 06:03:13,754 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c37a9918a3717aa65d37a4 2026-03-27 06:03:13,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:03:13,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c37a9918a3717aa65d37a4 HTTP/1.1" 200 None 2026-03-27 06:03:13,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:03:13,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-27 06:03:13,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:03:13,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c37a9918a3717aa65d37a4 HTTP/1.1" 200 None 2026-03-27 06:03:13,845 - lava_callback - ERROR - Error processing callback for node 69c37a9918a3717aa65d37a4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38162 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41028 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35460 - "GET / HTTP/1.0" 200 OK 2026-03-27 06:04:33,815 - lava_callback - INFO - Received callback for node 69c37ae718a3717aa65d37ae INFO: 172.20.0.1:54854 - "POST /node/69c37ae718a3717aa65d37ae HTTP/1.0" 202 Accepted 2026-03-27 06:04:33,821 - lava_callback - INFO - Starting async job submit for node 69c37ae718a3717aa65d37ae 2026-03-27 06:04:33,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:04:33,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c37ae718a3717aa65d37ae HTTP/1.1" 200 None 2026-03-27 06:04:33,837 - lava_callback - DEBUG - Processing node 69c37ae718a3717aa65d37ae: baseline-arm64-clabbe 2026-03-27 06:04:33,838 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c37ae718a3717aa65d37ae 2026-03-27 06:04:33,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:04:33,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c37ae718a3717aa65d37ae HTTP/1.1" 200 None 2026-03-27 06:04:33,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:04:33,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-27 06:04:33,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:04:33,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c37ae718a3717aa65d37ae HTTP/1.1" 200 None 2026-03-27 06:04:33,890 - lava_callback - ERROR - Error processing callback for node 69c37ae718a3717aa65d37ae: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47256 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51170 - "GET / HTTP/1.0" 200 OK 2026-03-27 06:05:53,844 - lava_callback - INFO - Received callback for node 69c37b3318a3717aa65d37be INFO: 172.20.0.1:42408 - "POST /node/69c37b3318a3717aa65d37be HTTP/1.0" 202 Accepted 2026-03-27 06:05:53,850 - lava_callback - INFO - Starting async job submit for node 69c37b3318a3717aa65d37be 2026-03-27 06:05:53,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:05:53,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c37b3318a3717aa65d37be HTTP/1.1" 200 None 2026-03-27 06:05:53,870 - lava_callback - DEBUG - Processing node 69c37b3318a3717aa65d37be: baseline-arm64-clabbe 2026-03-27 06:05:53,870 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c37b3318a3717aa65d37be 2026-03-27 06:05:53,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:05:53,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c37b3318a3717aa65d37be HTTP/1.1" 200 None 2026-03-27 06:05:53,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:05:53,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-27 06:05:53,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:05:53,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c37b3318a3717aa65d37be HTTP/1.1" 200 None 2026-03-27 06:05:53,933 - lava_callback - ERROR - Error processing callback for node 69c37b3318a3717aa65d37be: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38668 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39628 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36898 - "GET / HTTP/1.0" 200 OK 2026-03-27 06:08:13,976 - lava_callback - INFO - Received callback for node 69c37bcb18a3717aa65d37d5 INFO: 172.20.0.1:36414 - "POST /node/69c37bcb18a3717aa65d37d5 HTTP/1.0" 202 Accepted 2026-03-27 06:08:13,982 - lava_callback - INFO - Starting async job submit for node 69c37bcb18a3717aa65d37d5 2026-03-27 06:08:13,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:08:14,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c37bcb18a3717aa65d37d5 HTTP/1.1" 200 None 2026-03-27 06:08:14,005 - lava_callback - DEBUG - Processing node 69c37bcb18a3717aa65d37d5: baseline-arm64-clabbe 2026-03-27 06:08:14,005 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c37bcb18a3717aa65d37d5 2026-03-27 06:08:14,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:08:14,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c37bcb18a3717aa65d37d5 HTTP/1.1" 200 None 2026-03-27 06:08:14,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:08:14,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-27 06:08:14,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:08:14,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c37bcb18a3717aa65d37d5 HTTP/1.1" 200 None 2026-03-27 06:08:14,064 - lava_callback - ERROR - Error processing callback for node 69c37bcb18a3717aa65d37d5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36426 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55420 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55430 - "GET / HTTP/1.0" 200 OK 2026-03-27 06:09:25,109 - lava_callback - INFO - Received callback for node 69c37c0e18a3717aa65d37e3 INFO: 172.20.0.1:36944 - "POST /node/69c37c0e18a3717aa65d37e3 HTTP/1.0" 202 Accepted 2026-03-27 06:09:25,114 - lava_callback - INFO - Starting async job submit for node 69c37c0e18a3717aa65d37e3 2026-03-27 06:09:25,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:09:25,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c37c0e18a3717aa65d37e3 HTTP/1.1" 200 None 2026-03-27 06:09:25,133 - lava_callback - DEBUG - Processing node 69c37c0e18a3717aa65d37e3: baseline-arm64-baylibre 2026-03-27 06:09:25,133 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c37c0e18a3717aa65d37e3 2026-03-27 06:09:25,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:09:25,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c37c0e18a3717aa65d37e3 HTTP/1.1" 200 None 2026-03-27 06:09:25,171 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:09:25,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-27 06:09:25,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:09:25,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c37c0e18a3717aa65d37e3 HTTP/1.1" 200 None 2026-03-27 06:09:25,216 - lava_callback - ERROR - Error processing callback for node 69c37c0e18a3717aa65d37e3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56188 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45204 - "GET / HTTP/1.0" 200 OK 2026-03-27 06:10:43,943 - lava_callback - INFO - Received callback for node 69c5cd7b2cb62322ad965dea INFO: 172.20.0.1:41590 - "POST /node/69c5cd7b2cb62322ad965dea HTTP/1.0" 202 Accepted 2026-03-27 06:10:43,950 - lava_callback - INFO - Starting async job submit for node 69c5cd7b2cb62322ad965dea 2026-03-27 06:10:43,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:10:44,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cd7b2cb62322ad965dea HTTP/1.1" 200 None 2026-03-27 06:10:44,011 - lava_callback - DEBUG - Processing node 69c5cd7b2cb62322ad965dea: baseline-nfs-arm64 2026-03-27 06:10:44,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 06:10:44,685 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 06:10:44,686 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5cd7b2cb62322ad965dea/log.txt.gz 2026-03-27 06:10:44,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 06:10:45,155 - lava_callback - INFO - Received callback for node 69c37c5a18a3717aa65d37ee INFO: 172.20.0.1:41594 - "POST /node/69c37c5a18a3717aa65d37ee HTTP/1.0" 202 Accepted 2026-03-27 06:10:45,161 - lava_callback - INFO - Starting async job submit for node 69c37c5a18a3717aa65d37ee 2026-03-27 06:10:45,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:10:45,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c37c5a18a3717aa65d37ee HTTP/1.1" 200 None 2026-03-27 06:10:45,179 - lava_callback - DEBUG - Processing node 69c37c5a18a3717aa65d37ee: baseline-arm64-baylibre 2026-03-27 06:10:45,179 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c37c5a18a3717aa65d37ee 2026-03-27 06:10:45,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:10:45,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c37c5a18a3717aa65d37ee HTTP/1.1" 200 None 2026-03-27 06:10:45,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:10:45,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-27 06:10:45,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:10:45,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c37c5a18a3717aa65d37ee HTTP/1.1" 200 None 2026-03-27 06:10:45,243 - lava_callback - ERROR - Error processing callback for node 69c37c5a18a3717aa65d37ee: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-27 06:10:45,487 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 06:10:45,489 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5cd7b2cb62322ad965dea/lava-logs.yaml 2026-03-27 06:10:45,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 06:10:45,682 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 06:10:45,683 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5cd7b2cb62322ad965dea/lava_callback.json.gz 2026-03-27 06:10:45,711 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:10:45,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cd7b2cb62322ad965dea HTTP/1.1" 200 None 2026-03-27 06:10:45,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:10:45,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-27 06:10:45,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:10:45,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5cd7b2cb62322ad965dea HTTP/1.1" 200 None 2026-03-27 06:10:45,819 - lava_callback - INFO - Completed processing callback for node 69c5cd7b2cb62322ad965dea INFO: 172.20.0.1:41606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35550 - "GET / HTTP/1.0" 200 OK 2026-03-27 06:11:01,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:11:01,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:51452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51788 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56946 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47346 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44132 - "GET / HTTP/1.0" 200 OK 2026-03-27 06:13:34,218 - lava_callback - INFO - Received callback for node 69c37d0118a3717aa65d3804 INFO: 172.20.0.1:53486 - "POST /node/69c37d0118a3717aa65d3804 HTTP/1.0" 202 Accepted 2026-03-27 06:13:34,223 - lava_callback - INFO - Starting async job submit for node 69c37d0118a3717aa65d3804 2026-03-27 06:13:34,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:13:34,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c37d0118a3717aa65d3804 HTTP/1.1" 200 None 2026-03-27 06:13:34,241 - lava_callback - DEBUG - Processing node 69c37d0118a3717aa65d3804: baseline-arm64-clabbe 2026-03-27 06:13:34,241 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c37d0118a3717aa65d3804 2026-03-27 06:13:34,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:13:34,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c37d0118a3717aa65d3804 HTTP/1.1" 200 None 2026-03-27 06:13:34,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:13:34,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-27 06:13:34,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:13:34,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c37d0118a3717aa65d3804 HTTP/1.1" 200 None 2026-03-27 06:13:34,320 - lava_callback - ERROR - Error processing callback for node 69c37d0118a3717aa65d3804: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43176 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48970 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41854 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53034 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34942 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54324 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37018 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33430 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58262 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58136 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58096 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51424 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36266 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38026 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49968 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47194 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56162 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50530 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57746 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39308 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36252 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43172 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60894 - "GET / HTTP/1.0" 200 OK 2026-03-27 06:36:06,607 - lava_callback - INFO - Received callback for node 69c3824a18a3717aa65d3867 INFO: 172.20.0.1:60906 - "POST /node/69c3824a18a3717aa65d3867 HTTP/1.0" 202 Accepted 2026-03-27 06:36:06,613 - lava_callback - INFO - Starting async job submit for node 69c3824a18a3717aa65d3867 2026-03-27 06:36:06,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:36:06,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3824a18a3717aa65d3867 HTTP/1.1" 200 None 2026-03-27 06:36:06,628 - lava_callback - DEBUG - Processing node 69c3824a18a3717aa65d3867: baseline-arm-baylibre 2026-03-27 06:36:06,629 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3824a18a3717aa65d3867 2026-03-27 06:36:06,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:36:06,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3824a18a3717aa65d3867 HTTP/1.1" 200 None 2026-03-27 06:36:06,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:36:06,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-27 06:36:06,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:36:06,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3824a18a3717aa65d3867 HTTP/1.1" 200 None 2026-03-27 06:36:06,694 - lava_callback - ERROR - Error processing callback for node 69c3824a18a3717aa65d3867: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38946 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54046 - "GET / HTTP/1.0" 200 OK 2026-03-27 06:36:46,546 - lava_callback - INFO - Received callback for node 69c3826e18a3717aa65d3872 INFO: 172.20.0.1:54062 - "POST /node/69c3826e18a3717aa65d3872 HTTP/1.0" 202 Accepted 2026-03-27 06:36:46,552 - lava_callback - INFO - Starting async job submit for node 69c3826e18a3717aa65d3872 2026-03-27 06:36:46,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:36:46,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3826e18a3717aa65d3872 HTTP/1.1" 200 None 2026-03-27 06:36:46,568 - lava_callback - DEBUG - Processing node 69c3826e18a3717aa65d3872: baseline-arm-baylibre 2026-03-27 06:36:46,569 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3826e18a3717aa65d3872 2026-03-27 06:36:46,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:36:46,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3826e18a3717aa65d3872 HTTP/1.1" 200 None 2026-03-27 06:36:46,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:36:46,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-27 06:36:46,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:36:46,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3826e18a3717aa65d3872 HTTP/1.1" 200 None 2026-03-27 06:36:46,627 - lava_callback - ERROR - Error processing callback for node 69c3826e18a3717aa65d3872: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49204 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35108 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35748 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55116 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-27 06:40:15,469 - lava_callback - INFO - Received callback for node 69c3834518a3717aa65d38bf INFO: 172.20.0.1:55132 - "POST /node/69c3834518a3717aa65d38bf HTTP/1.0" 202 Accepted 2026-03-27 06:40:15,474 - lava_callback - INFO - Starting async job submit for node 69c3834518a3717aa65d38bf 2026-03-27 06:40:15,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:40:15,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3834518a3717aa65d38bf HTTP/1.1" 200 None 2026-03-27 06:40:15,497 - lava_callback - DEBUG - Processing node 69c3834518a3717aa65d38bf: baseline-arm-clabbe 2026-03-27 06:40:15,497 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3834518a3717aa65d38bf 2026-03-27 06:40:15,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:40:15,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3834518a3717aa65d38bf HTTP/1.1" 200 None 2026-03-27 06:40:15,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:40:15,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-27 06:40:15,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:40:15,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3834518a3717aa65d38bf HTTP/1.1" 200 None 2026-03-27 06:40:15,738 - lava_callback - ERROR - Error processing callback for node 69c3834518a3717aa65d38bf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52078 - "GET / HTTP/1.0" 200 OK 2026-03-27 06:40:55,386 - lava_callback - INFO - Received callback for node 69c3836918a3717aa65d38d0 INFO: 172.20.0.1:41512 - "POST /node/69c3836918a3717aa65d38d0 HTTP/1.0" 202 Accepted 2026-03-27 06:40:55,392 - lava_callback - INFO - Starting async job submit for node 69c3836918a3717aa65d38d0 2026-03-27 06:40:55,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:40:55,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3836918a3717aa65d38d0 HTTP/1.1" 200 None 2026-03-27 06:40:55,408 - lava_callback - DEBUG - Processing node 69c3836918a3717aa65d38d0: baseline-arm-clabbe 2026-03-27 06:40:55,408 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3836918a3717aa65d38d0 2026-03-27 06:40:55,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:40:55,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3836918a3717aa65d38d0 HTTP/1.1" 200 None 2026-03-27 06:40:55,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:40:55,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-27 06:40:55,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:40:55,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3836918a3717aa65d38d0 HTTP/1.1" 200 None 2026-03-27 06:40:55,472 - lava_callback - ERROR - Error processing callback for node 69c3836918a3717aa65d38d0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32812 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45714 - "GET / HTTP/1.0" 200 OK 2026-03-27 06:41:35,466 - lava_callback - INFO - Received callback for node 69c3838e18a3717aa65d38e1 INFO: 172.20.0.1:43160 - "POST /node/69c3838e18a3717aa65d38e1 HTTP/1.0" 202 Accepted 2026-03-27 06:41:35,471 - lava_callback - INFO - Starting async job submit for node 69c3838e18a3717aa65d38e1 2026-03-27 06:41:35,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:41:35,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3838e18a3717aa65d38e1 HTTP/1.1" 200 None 2026-03-27 06:41:35,487 - lava_callback - DEBUG - Processing node 69c3838e18a3717aa65d38e1: baseline-arm-clabbe 2026-03-27 06:41:35,488 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3838e18a3717aa65d38e1 2026-03-27 06:41:35,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:41:35,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3838e18a3717aa65d38e1 HTTP/1.1" 200 None 2026-03-27 06:41:35,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:41:35,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-27 06:41:35,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:41:35,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3838e18a3717aa65d38e1 HTTP/1.1" 200 None 2026-03-27 06:41:35,550 - lava_callback - ERROR - Error processing callback for node 69c3838e18a3717aa65d38e1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33786 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-27 06:42:15,445 - lava_callback - INFO - Received callback for node 69c383b318a3717aa65d38f8 INFO: 172.20.0.1:33788 - "POST /node/69c383b318a3717aa65d38f8 HTTP/1.0" 202 Accepted 2026-03-27 06:42:15,451 - lava_callback - INFO - Starting async job submit for node 69c383b318a3717aa65d38f8 2026-03-27 06:42:15,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:42:15,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c383b318a3717aa65d38f8 HTTP/1.1" 200 None 2026-03-27 06:42:15,466 - lava_callback - DEBUG - Processing node 69c383b318a3717aa65d38f8: baseline-arm-clabbe 2026-03-27 06:42:15,466 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c383b318a3717aa65d38f8 2026-03-27 06:42:15,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:42:15,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c383b318a3717aa65d38f8 HTTP/1.1" 200 None 2026-03-27 06:42:15,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:42:15,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-27 06:42:15,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:42:15,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c383b318a3717aa65d38f8 HTTP/1.1" 200 None 2026-03-27 06:42:15,520 - lava_callback - ERROR - Error processing callback for node 69c383b318a3717aa65d38f8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32888 - "GET / HTTP/1.0" 200 OK 2026-03-27 06:42:35,539 - lava_callback - INFO - Received callback for node 69c383d818a3717aa65d3901 INFO: 172.20.0.1:52106 - "POST /node/69c383d818a3717aa65d3901 HTTP/1.0" 202 Accepted 2026-03-27 06:42:35,545 - lava_callback - INFO - Starting async job submit for node 69c383d818a3717aa65d3901 2026-03-27 06:42:35,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:42:35,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c383d818a3717aa65d3901 HTTP/1.1" 200 None 2026-03-27 06:42:35,562 - lava_callback - DEBUG - Processing node 69c383d818a3717aa65d3901: baseline-arm-clabbe 2026-03-27 06:42:35,562 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c383d818a3717aa65d3901 2026-03-27 06:42:35,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:42:35,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c383d818a3717aa65d3901 HTTP/1.1" 200 None 2026-03-27 06:42:35,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:42:35,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-27 06:42:35,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:42:35,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c383d818a3717aa65d3901 HTTP/1.1" 200 None 2026-03-27 06:42:35,627 - lava_callback - ERROR - Error processing callback for node 69c383d818a3717aa65d3901: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52848 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-27 06:43:15,483 - lava_callback - INFO - Received callback for node 69c383fd18a3717aa65d3910 INFO: 172.20.0.1:52862 - "POST /node/69c383fd18a3717aa65d3910 HTTP/1.0" 202 Accepted 2026-03-27 06:43:15,488 - lava_callback - INFO - Starting async job submit for node 69c383fd18a3717aa65d3910 2026-03-27 06:43:15,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:43:15,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c383fd18a3717aa65d3910 HTTP/1.1" 200 None 2026-03-27 06:43:15,504 - lava_callback - DEBUG - Processing node 69c383fd18a3717aa65d3910: baseline-arm-clabbe 2026-03-27 06:43:15,504 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c383fd18a3717aa65d3910 2026-03-27 06:43:15,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:43:15,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c383fd18a3717aa65d3910 HTTP/1.1" 200 None 2026-03-27 06:43:15,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:43:15,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-27 06:43:15,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:43:15,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c383fd18a3717aa65d3910 HTTP/1.1" 200 None 2026-03-27 06:43:15,563 - lava_callback - ERROR - Error processing callback for node 69c383fd18a3717aa65d3910: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33676 - "GET / HTTP/1.0" 200 OK 2026-03-27 06:43:55,566 - lava_callback - INFO - Received callback for node 69c3842218a3717aa65d391f INFO: 172.20.0.1:35770 - "POST /node/69c3842218a3717aa65d391f HTTP/1.0" 202 Accepted 2026-03-27 06:43:55,571 - lava_callback - INFO - Starting async job submit for node 69c3842218a3717aa65d391f 2026-03-27 06:43:55,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:43:55,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3842218a3717aa65d391f HTTP/1.1" 200 None 2026-03-27 06:43:55,587 - lava_callback - DEBUG - Processing node 69c3842218a3717aa65d391f: baseline-arm-clabbe 2026-03-27 06:43:55,587 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3842218a3717aa65d391f 2026-03-27 06:43:55,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:43:55,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3842218a3717aa65d391f HTTP/1.1" 200 None 2026-03-27 06:43:55,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:43:55,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-27 06:43:55,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:43:55,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3842218a3717aa65d391f HTTP/1.1" 200 None 2026-03-27 06:43:55,644 - lava_callback - ERROR - Error processing callback for node 69c3842218a3717aa65d391f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52614 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37962 - "GET / HTTP/1.0" 200 OK 2026-03-27 06:44:35,527 - lava_callback - INFO - Received callback for node 69c3844718a3717aa65d3930 INFO: 172.20.0.1:48004 - "POST /node/69c3844718a3717aa65d3930 HTTP/1.0" 202 Accepted 2026-03-27 06:44:35,532 - lava_callback - INFO - Starting async job submit for node 69c3844718a3717aa65d3930 2026-03-27 06:44:35,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:44:35,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3844718a3717aa65d3930 HTTP/1.1" 200 None 2026-03-27 06:44:35,548 - lava_callback - DEBUG - Processing node 69c3844718a3717aa65d3930: baseline-arm-clabbe 2026-03-27 06:44:35,548 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3844718a3717aa65d3930 2026-03-27 06:44:35,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:44:35,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3844718a3717aa65d3930 HTTP/1.1" 200 None 2026-03-27 06:44:35,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:44:35,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-27 06:44:35,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:44:35,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3844718a3717aa65d3930 HTTP/1.1" 200 None 2026-03-27 06:44:35,612 - lava_callback - ERROR - Error processing callback for node 69c3844718a3717aa65d3930: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35928 - "GET / HTTP/1.0" 200 OK 2026-03-27 06:45:07,127 - lava_callback - INFO - Received callback for node 69c3846618a3717aa65d3940 INFO: 172.20.0.1:35944 - "POST /node/69c3846618a3717aa65d3940 HTTP/1.0" 202 Accepted 2026-03-27 06:45:07,133 - lava_callback - INFO - Starting async job submit for node 69c3846618a3717aa65d3940 2026-03-27 06:45:07,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:45:07,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3846618a3717aa65d3940 HTTP/1.1" 200 None 2026-03-27 06:45:07,153 - lava_callback - DEBUG - Processing node 69c3846618a3717aa65d3940: baseline-arm-baylibre 2026-03-27 06:45:07,154 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3846618a3717aa65d3940 2026-03-27 06:45:07,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:45:07,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3846618a3717aa65d3940 HTTP/1.1" 200 None 2026-03-27 06:45:07,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:45:07,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-27 06:45:07,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:45:07,204 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3846618a3717aa65d3940 HTTP/1.1" 200 None 2026-03-27 06:45:07,210 - lava_callback - ERROR - Error processing callback for node 69c3846618a3717aa65d3940: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52678 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55208 - "GET / HTTP/1.0" 200 OK 2026-03-27 06:45:47,054 - lava_callback - INFO - Received callback for node 69c3848918a3717aa65d394e INFO: 172.20.0.1:55218 - "POST /node/69c3848918a3717aa65d394e HTTP/1.0" 202 Accepted 2026-03-27 06:45:47,062 - lava_callback - INFO - Starting async job submit for node 69c3848918a3717aa65d394e 2026-03-27 06:45:47,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:45:47,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3848918a3717aa65d394e HTTP/1.1" 200 None 2026-03-27 06:45:47,079 - lava_callback - DEBUG - Processing node 69c3848918a3717aa65d394e: baseline-arm-baylibre 2026-03-27 06:45:47,079 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3848918a3717aa65d394e 2026-03-27 06:45:47,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:45:47,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3848918a3717aa65d394e HTTP/1.1" 200 None 2026-03-27 06:45:47,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:45:47,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-27 06:45:47,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:45:47,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3848918a3717aa65d394e HTTP/1.1" 200 None 2026-03-27 06:45:47,154 - lava_callback - ERROR - Error processing callback for node 69c3848918a3717aa65d394e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58248 - "GET / HTTP/1.0" 200 OK 2026-03-27 06:46:07,041 - lava_callback - INFO - Received callback for node 69c384ad18a3717aa65d395e INFO: 172.20.0.1:58264 - "POST /node/69c384ad18a3717aa65d395e HTTP/1.0" 202 Accepted 2026-03-27 06:46:07,047 - lava_callback - INFO - Starting async job submit for node 69c384ad18a3717aa65d395e 2026-03-27 06:46:07,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:46:07,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c384ad18a3717aa65d395e HTTP/1.1" 200 None 2026-03-27 06:46:07,078 - lava_callback - DEBUG - Processing node 69c384ad18a3717aa65d395e: baseline-arm-baylibre 2026-03-27 06:46:07,078 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c384ad18a3717aa65d395e 2026-03-27 06:46:07,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:46:07,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c384ad18a3717aa65d395e HTTP/1.1" 200 None 2026-03-27 06:46:07,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:46:07,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-27 06:46:07,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:46:07,130 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c384ad18a3717aa65d395e HTTP/1.1" 200 None 2026-03-27 06:46:07,131 - lava_callback - ERROR - Error processing callback for node 69c384ad18a3717aa65d395e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58828 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38332 - "GET / HTTP/1.0" 200 OK 2026-03-27 06:46:47,148 - lava_callback - INFO - Received callback for node 69c384d118a3717aa65d396e INFO: 172.20.0.1:38348 - "POST /node/69c384d118a3717aa65d396e HTTP/1.0" 202 Accepted 2026-03-27 06:46:47,153 - lava_callback - INFO - Starting async job submit for node 69c384d118a3717aa65d396e 2026-03-27 06:46:47,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:46:47,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c384d118a3717aa65d396e HTTP/1.1" 200 None 2026-03-27 06:46:47,170 - lava_callback - DEBUG - Processing node 69c384d118a3717aa65d396e: baseline-arm-baylibre 2026-03-27 06:46:47,170 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c384d118a3717aa65d396e 2026-03-27 06:46:47,171 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:46:47,183 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c384d118a3717aa65d396e HTTP/1.1" 200 None 2026-03-27 06:46:47,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:46:47,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-27 06:46:47,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:46:47,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c384d118a3717aa65d396e HTTP/1.1" 200 None 2026-03-27 06:46:47,221 - lava_callback - ERROR - Error processing callback for node 69c384d118a3717aa65d396e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33658 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55620 - "GET / HTTP/1.0" 200 OK 2026-03-27 06:47:35,775 - lava_callback - INFO - Received callback for node 69c384f718a3717aa65d3983 INFO: 172.20.0.1:47290 - "POST /node/69c384f718a3717aa65d3983 HTTP/1.0" 202 Accepted 2026-03-27 06:47:35,781 - lava_callback - INFO - Starting async job submit for node 69c384f718a3717aa65d3983 2026-03-27 06:47:35,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:47:35,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c384f718a3717aa65d3983 HTTP/1.1" 200 None 2026-03-27 06:47:35,797 - lava_callback - DEBUG - Processing node 69c384f718a3717aa65d3983: baseline-arm-clabbe 2026-03-27 06:47:35,797 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c384f718a3717aa65d3983 2026-03-27 06:47:35,798 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:47:35,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c384f718a3717aa65d3983 HTTP/1.1" 200 None 2026-03-27 06:47:35,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:47:35,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-27 06:47:35,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:47:35,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c384f718a3717aa65d3983 HTTP/1.1" 200 None 2026-03-27 06:47:35,848 - lava_callback - ERROR - Error processing callback for node 69c384f718a3717aa65d3983: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36772 - "GET / HTTP/1.0" 200 OK 2026-03-27 06:48:07,212 - lava_callback - INFO - Received callback for node 69c3851918a3717aa65d398f INFO: 172.20.0.1:36788 - "POST /node/69c3851918a3717aa65d398f HTTP/1.0" 202 Accepted 2026-03-27 06:48:07,217 - lava_callback - INFO - Starting async job submit for node 69c3851918a3717aa65d398f 2026-03-27 06:48:07,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:48:07,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3851918a3717aa65d398f HTTP/1.1" 200 None 2026-03-27 06:48:07,234 - lava_callback - DEBUG - Processing node 69c3851918a3717aa65d398f: baseline-arm-baylibre 2026-03-27 06:48:07,234 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3851918a3717aa65d398f 2026-03-27 06:48:07,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:48:07,247 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3851918a3717aa65d398f HTTP/1.1" 200 None 2026-03-27 06:48:07,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:48:07,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-27 06:48:07,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:48:07,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3851918a3717aa65d398f HTTP/1.1" 200 None 2026-03-27 06:48:07,284 - lava_callback - ERROR - Error processing callback for node 69c3851918a3717aa65d398f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46476 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58222 - "GET / HTTP/1.0" 200 OK 2026-03-27 06:48:47,218 - lava_callback - INFO - Received callback for node 69c3853d18a3717aa65d39a2 INFO: 172.20.0.1:58224 - "POST /node/69c3853d18a3717aa65d39a2 HTTP/1.0" 202 Accepted 2026-03-27 06:48:47,224 - lava_callback - INFO - Starting async job submit for node 69c3853d18a3717aa65d39a2 2026-03-27 06:48:47,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:48:47,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3853d18a3717aa65d39a2 HTTP/1.1" 200 None 2026-03-27 06:48:47,240 - lava_callback - DEBUG - Processing node 69c3853d18a3717aa65d39a2: baseline-arm-baylibre 2026-03-27 06:48:47,240 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3853d18a3717aa65d39a2 2026-03-27 06:48:47,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:48:47,254 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3853d18a3717aa65d39a2 HTTP/1.1" 200 None 2026-03-27 06:48:47,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:48:47,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-27 06:48:47,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:48:47,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3853d18a3717aa65d39a2 HTTP/1.1" 200 None 2026-03-27 06:48:47,302 - lava_callback - ERROR - Error processing callback for node 69c3853d18a3717aa65d39a2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51274 - "GET / HTTP/1.0" 200 OK 2026-03-27 06:49:03,712 - lava_callback - INFO - Received callback for node 69c5d5542cb62322ad96625f INFO: 172.20.0.1:55082 - "POST /node/69c5d5542cb62322ad96625f HTTP/1.0" 202 Accepted 2026-03-27 06:49:03,720 - lava_callback - INFO - Starting async job submit for node 69c5d5542cb62322ad96625f 2026-03-27 06:49:03,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:49:03,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5542cb62322ad96625f HTTP/1.1" 200 None 2026-03-27 06:49:03,762 - lava_callback - DEBUG - Processing node 69c5d5542cb62322ad96625f: baseline-arm64 2026-03-27 06:49:04,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 06:49:04,554 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 06:49:04,556 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5d5542cb62322ad96625f/log.txt.gz 2026-03-27 06:49:04,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 06:49:05,021 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 06:49:05,023 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5d5542cb62322ad96625f/lava-logs.yaml 2026-03-27 06:49:05,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 06:49:05,223 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 06:49:05,225 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5d5542cb62322ad96625f/lava_callback.json.gz 2026-03-27 06:49:05,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:49:05,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5542cb62322ad96625f HTTP/1.1" 200 None 2026-03-27 06:49:05,266 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:49:05,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-27 06:49:05,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:49:05,310 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d5542cb62322ad96625f HTTP/1.1" 200 None 2026-03-27 06:49:05,360 - lava_callback - INFO - Completed processing callback for node 69c5d5542cb62322ad96625f INFO: 172.20.0.1:55090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45498 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-27 06:49:15,834 - lava_callback - INFO - Received callback for node 69c3856218a3717aa65d39b4 INFO: 172.20.0.1:45500 - "POST /node/69c3856218a3717aa65d39b4 HTTP/1.0" 202 Accepted 2026-03-27 06:49:15,839 - lava_callback - INFO - Starting async job submit for node 69c3856218a3717aa65d39b4 2026-03-27 06:49:15,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:49:15,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3856218a3717aa65d39b4 HTTP/1.1" 200 None 2026-03-27 06:49:15,855 - lava_callback - DEBUG - Processing node 69c3856218a3717aa65d39b4: baseline-arm-clabbe 2026-03-27 06:49:15,856 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3856218a3717aa65d39b4 2026-03-27 06:49:15,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:49:15,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3856218a3717aa65d39b4 HTTP/1.1" 200 None 2026-03-27 06:49:15,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:49:15,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-27 06:49:15,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:49:15,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3856218a3717aa65d39b4 HTTP/1.1" 200 None 2026-03-27 06:49:15,912 - lava_callback - ERROR - Error processing callback for node 69c3856218a3717aa65d39b4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49868 - "GET / HTTP/1.0" 200 OK 2026-03-27 06:49:31,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:49:31,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:37824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54722 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38536 - "GET / HTTP/1.0" 200 OK 2026-03-27 06:50:55,884 - lava_callback - INFO - Received callback for node 69c385ca18a3717aa65d39e4 INFO: 172.20.0.1:49740 - "POST /node/69c385ca18a3717aa65d39e4 HTTP/1.0" 202 Accepted 2026-03-27 06:50:55,889 - lava_callback - INFO - Starting async job submit for node 69c385ca18a3717aa65d39e4 2026-03-27 06:50:55,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:50:55,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c385ca18a3717aa65d39e4 HTTP/1.1" 200 None 2026-03-27 06:50:55,905 - lava_callback - DEBUG - Processing node 69c385ca18a3717aa65d39e4: baseline-arm-clabbe 2026-03-27 06:50:55,906 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c385ca18a3717aa65d39e4 2026-03-27 06:50:55,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:50:55,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c385ca18a3717aa65d39e4 HTTP/1.1" 200 None 2026-03-27 06:50:55,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:50:55,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-27 06:50:55,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:50:55,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c385ca18a3717aa65d39e4 HTTP/1.1" 200 None 2026-03-27 06:50:55,963 - lava_callback - ERROR - Error processing callback for node 69c385ca18a3717aa65d39e4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35988 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56264 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-27 06:52:15,989 - lava_callback - INFO - Received callback for node 69c3861118a3717aa65d3a0a INFO: 172.20.0.1:56268 - "POST /node/69c3861118a3717aa65d3a0a HTTP/1.0" 202 Accepted 2026-03-27 06:52:15,995 - lava_callback - INFO - Starting async job submit for node 69c3861118a3717aa65d3a0a 2026-03-27 06:52:15,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:52:16,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3861118a3717aa65d3a0a HTTP/1.1" 200 None 2026-03-27 06:52:16,018 - lava_callback - DEBUG - Processing node 69c3861118a3717aa65d3a0a: baseline-arm-clabbe 2026-03-27 06:52:16,018 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3861118a3717aa65d3a0a 2026-03-27 06:52:16,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:52:16,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3861118a3717aa65d3a0a HTTP/1.1" 200 None 2026-03-27 06:52:16,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:52:16,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-27 06:52:16,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:52:16,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3861118a3717aa65d3a0a HTTP/1.1" 200 None 2026-03-27 06:52:16,082 - lava_callback - ERROR - Error processing callback for node 69c3861118a3717aa65d3a0a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33038 - "GET / HTTP/1.0" 200 OK 2026-03-27 06:52:55,944 - lava_callback - INFO - Received callback for node 69c3863518a3717aa65d3a1c INFO: 172.20.0.1:48960 - "POST /node/69c3863518a3717aa65d3a1c HTTP/1.0" 202 Accepted 2026-03-27 06:52:55,949 - lava_callback - INFO - Starting async job submit for node 69c3863518a3717aa65d3a1c 2026-03-27 06:52:55,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:52:55,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3863518a3717aa65d3a1c HTTP/1.1" 200 None 2026-03-27 06:52:55,966 - lava_callback - DEBUG - Processing node 69c3863518a3717aa65d3a1c: baseline-arm-clabbe 2026-03-27 06:52:55,966 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3863518a3717aa65d3a1c 2026-03-27 06:52:55,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:52:55,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3863518a3717aa65d3a1c HTTP/1.1" 200 None 2026-03-27 06:52:55,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:52:55,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-27 06:52:55,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:52:56,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3863518a3717aa65d3a1c HTTP/1.1" 200 None 2026-03-27 06:52:56,023 - lava_callback - ERROR - Error processing callback for node 69c3863518a3717aa65d3a1c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33560 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-27 06:53:15,974 - lava_callback - INFO - Received callback for node 69c3865918a3717aa65d3a31 INFO: 172.20.0.1:33574 - "POST /node/69c3865918a3717aa65d3a31 HTTP/1.0" 202 Accepted 2026-03-27 06:53:15,980 - lava_callback - INFO - Starting async job submit for node 69c3865918a3717aa65d3a31 2026-03-27 06:53:15,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:53:15,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3865918a3717aa65d3a31 HTTP/1.1" 200 None 2026-03-27 06:53:15,997 - lava_callback - DEBUG - Processing node 69c3865918a3717aa65d3a31: baseline-arm-clabbe 2026-03-27 06:53:15,997 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3865918a3717aa65d3a31 2026-03-27 06:53:15,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:53:16,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3865918a3717aa65d3a31 HTTP/1.1" 200 None 2026-03-27 06:53:16,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:53:16,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-27 06:53:16,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:53:16,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3865918a3717aa65d3a31 HTTP/1.1" 200 None 2026-03-27 06:53:16,060 - lava_callback - ERROR - Error processing callback for node 69c3865918a3717aa65d3a31: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38426 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48126 - "GET / HTTP/1.0" 200 OK 2026-03-27 06:54:36,080 - lava_callback - INFO - Received callback for node 69c3869f18a3717aa65d3a54 INFO: 172.20.0.1:55992 - "POST /node/69c3869f18a3717aa65d3a54 HTTP/1.0" 202 Accepted 2026-03-27 06:54:36,086 - lava_callback - INFO - Starting async job submit for node 69c3869f18a3717aa65d3a54 2026-03-27 06:54:36,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:54:36,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3869f18a3717aa65d3a54 HTTP/1.1" 200 None 2026-03-27 06:54:36,105 - lava_callback - DEBUG - Processing node 69c3869f18a3717aa65d3a54: baseline-arm-clabbe 2026-03-27 06:54:36,105 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3869f18a3717aa65d3a54 2026-03-27 06:54:36,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:54:36,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3869f18a3717aa65d3a54 HTTP/1.1" 200 None 2026-03-27 06:54:36,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:54:36,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-27 06:54:36,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:54:36,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3869f18a3717aa65d3a54 HTTP/1.1" 200 None 2026-03-27 06:54:36,183 - lava_callback - ERROR - Error processing callback for node 69c3869f18a3717aa65d3a54: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59426 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45170 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-27 06:56:16,077 - lava_callback - INFO - Received callback for node 69c3870918a3717aa65d3a87 INFO: 172.20.0.1:45174 - "POST /node/69c3870918a3717aa65d3a87 HTTP/1.0" 202 Accepted 2026-03-27 06:56:16,083 - lava_callback - INFO - Starting async job submit for node 69c3870918a3717aa65d3a87 2026-03-27 06:56:16,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:56:16,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3870918a3717aa65d3a87 HTTP/1.1" 200 None 2026-03-27 06:56:16,099 - lava_callback - DEBUG - Processing node 69c3870918a3717aa65d3a87: baseline-arm-clabbe 2026-03-27 06:56:16,099 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3870918a3717aa65d3a87 2026-03-27 06:56:16,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:56:16,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3870918a3717aa65d3a87 HTTP/1.1" 200 None 2026-03-27 06:56:16,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:56:16,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-27 06:56:16,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:56:16,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3870918a3717aa65d3a87 HTTP/1.1" 200 None 2026-03-27 06:56:16,180 - lava_callback - ERROR - Error processing callback for node 69c3870918a3717aa65d3a87: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37484 - "GET / HTTP/1.0" 200 OK 2026-03-27 06:56:56,078 - lava_callback - INFO - Received callback for node 69c3872d18a3717aa65d3a99 INFO: 172.20.0.1:49528 - "POST /node/69c3872d18a3717aa65d3a99 HTTP/1.0" 202 Accepted 2026-03-27 06:56:56,084 - lava_callback - INFO - Starting async job submit for node 69c3872d18a3717aa65d3a99 2026-03-27 06:56:56,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:56:56,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3872d18a3717aa65d3a99 HTTP/1.1" 200 None 2026-03-27 06:56:56,100 - lava_callback - DEBUG - Processing node 69c3872d18a3717aa65d3a99: baseline-arm-clabbe 2026-03-27 06:56:56,100 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3872d18a3717aa65d3a99 2026-03-27 06:56:56,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:56:56,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3872d18a3717aa65d3a99 HTTP/1.1" 200 None 2026-03-27 06:56:56,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:56:56,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-27 06:56:56,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:56:56,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3872d18a3717aa65d3a99 HTTP/1.1" 200 None 2026-03-27 06:56:56,157 - lava_callback - ERROR - Error processing callback for node 69c3872d18a3717aa65d3a99: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41688 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39898 - "GET / HTTP/1.0" 200 OK 2026-03-27 06:57:36,219 - lava_callback - INFO - Received callback for node 69c3875118a3717aa65d3aac INFO: 172.20.0.1:47226 - "POST /node/69c3875118a3717aa65d3aac HTTP/1.0" 202 Accepted 2026-03-27 06:57:36,225 - lava_callback - INFO - Starting async job submit for node 69c3875118a3717aa65d3aac 2026-03-27 06:57:36,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:57:36,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3875118a3717aa65d3aac HTTP/1.1" 200 None 2026-03-27 06:57:36,245 - lava_callback - DEBUG - Processing node 69c3875118a3717aa65d3aac: baseline-arm-clabbe 2026-03-27 06:57:36,245 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3875118a3717aa65d3aac 2026-03-27 06:57:36,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:57:36,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3875118a3717aa65d3aac HTTP/1.1" 200 None 2026-03-27 06:57:36,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:57:36,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-27 06:57:36,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:57:36,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3875118a3717aa65d3aac HTTP/1.1" 200 None 2026-03-27 06:57:36,320 - lava_callback - ERROR - Error processing callback for node 69c3875118a3717aa65d3aac: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33200 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-27 06:58:16,133 - lava_callback - INFO - Received callback for node 69c3877418a3717aa65d3abd INFO: 172.20.0.1:33204 - "POST /node/69c3877418a3717aa65d3abd HTTP/1.0" 202 Accepted 2026-03-27 06:58:16,139 - lava_callback - INFO - Starting async job submit for node 69c3877418a3717aa65d3abd 2026-03-27 06:58:16,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:58:16,154 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3877418a3717aa65d3abd HTTP/1.1" 200 None 2026-03-27 06:58:16,155 - lava_callback - DEBUG - Processing node 69c3877418a3717aa65d3abd: baseline-arm-clabbe 2026-03-27 06:58:16,155 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3877418a3717aa65d3abd 2026-03-27 06:58:16,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:58:16,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3877418a3717aa65d3abd HTTP/1.1" 200 None 2026-03-27 06:58:16,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:58:16,185 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-27 06:58:16,187 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:58:16,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3877418a3717aa65d3abd HTTP/1.1" 200 None 2026-03-27 06:58:16,227 - lava_callback - ERROR - Error processing callback for node 69c3877418a3717aa65d3abd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58818 - "GET / HTTP/1.0" 200 OK 2026-03-27 06:58:36,166 - lava_callback - INFO - Received callback for node 69c3879918a3717aa65d3ac5 INFO: 172.20.0.1:37350 - "POST /node/69c3879918a3717aa65d3ac5 HTTP/1.0" 202 Accepted 2026-03-27 06:58:36,172 - lava_callback - INFO - Starting async job submit for node 69c3879918a3717aa65d3ac5 2026-03-27 06:58:36,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:58:36,187 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3879918a3717aa65d3ac5 HTTP/1.1" 200 None 2026-03-27 06:58:36,188 - lava_callback - DEBUG - Processing node 69c3879918a3717aa65d3ac5: baseline-arm-clabbe 2026-03-27 06:58:36,189 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3879918a3717aa65d3ac5 2026-03-27 06:58:36,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:58:36,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3879918a3717aa65d3ac5 HTTP/1.1" 200 None 2026-03-27 06:58:36,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:58:36,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-27 06:58:36,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:58:36,247 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3879918a3717aa65d3ac5 HTTP/1.1" 200 None 2026-03-27 06:58:36,248 - lava_callback - ERROR - Error processing callback for node 69c3879918a3717aa65d3ac5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33960 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-27 06:59:16,179 - lava_callback - INFO - Received callback for node 69c387bd18a3717aa65d3ad0 INFO: 172.20.0.1:33972 - "POST /node/69c387bd18a3717aa65d3ad0 HTTP/1.0" 202 Accepted 2026-03-27 06:59:16,185 - lava_callback - INFO - Starting async job submit for node 69c387bd18a3717aa65d3ad0 2026-03-27 06:59:16,186 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:59:16,201 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c387bd18a3717aa65d3ad0 HTTP/1.1" 200 None 2026-03-27 06:59:16,202 - lava_callback - DEBUG - Processing node 69c387bd18a3717aa65d3ad0: baseline-arm-clabbe 2026-03-27 06:59:16,202 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c387bd18a3717aa65d3ad0 2026-03-27 06:59:16,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:59:16,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c387bd18a3717aa65d3ad0 HTTP/1.1" 200 None 2026-03-27 06:59:16,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:59:16,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-27 06:59:16,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:59:16,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c387bd18a3717aa65d3ad0 HTTP/1.1" 200 None 2026-03-27 06:59:16,262 - lava_callback - ERROR - Error processing callback for node 69c387bd18a3717aa65d3ad0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32834 - "GET / HTTP/1.0" 200 OK 2026-03-27 06:59:56,261 - lava_callback - INFO - Received callback for node 69c387e118a3717aa65d3adb INFO: 172.20.0.1:42544 - "POST /node/69c387e118a3717aa65d3adb HTTP/1.0" 202 Accepted 2026-03-27 06:59:56,268 - lava_callback - INFO - Starting async job submit for node 69c387e118a3717aa65d3adb 2026-03-27 06:59:56,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:59:56,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c387e118a3717aa65d3adb HTTP/1.1" 200 None 2026-03-27 06:59:56,284 - lava_callback - DEBUG - Processing node 69c387e118a3717aa65d3adb: baseline-arm-clabbe 2026-03-27 06:59:56,284 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c387e118a3717aa65d3adb 2026-03-27 06:59:56,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:59:56,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c387e118a3717aa65d3adb HTTP/1.1" 200 None 2026-03-27 06:59:56,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:59:56,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-27 06:59:56,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 06:59:56,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c387e118a3717aa65d3adb HTTP/1.1" 200 None 2026-03-27 06:59:56,341 - lava_callback - ERROR - Error processing callback for node 69c387e118a3717aa65d3adb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51698 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56288 - "GET / HTTP/1.0" 200 OK 2026-03-27 07:00:36,261 - lava_callback - INFO - Received callback for node 69c3880518a3717aa65d3aea INFO: 172.20.0.1:52494 - "POST /node/69c3880518a3717aa65d3aea HTTP/1.0" 202 Accepted 2026-03-27 07:00:36,267 - lava_callback - INFO - Starting async job submit for node 69c3880518a3717aa65d3aea 2026-03-27 07:00:36,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:00:36,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3880518a3717aa65d3aea HTTP/1.1" 200 None 2026-03-27 07:00:36,283 - lava_callback - DEBUG - Processing node 69c3880518a3717aa65d3aea: baseline-arm-clabbe 2026-03-27 07:00:36,283 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3880518a3717aa65d3aea 2026-03-27 07:00:36,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:00:36,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3880518a3717aa65d3aea HTTP/1.1" 200 None 2026-03-27 07:00:36,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:00:36,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-27 07:00:36,334 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:00:36,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3880518a3717aa65d3aea HTTP/1.1" 200 None 2026-03-27 07:00:36,356 - lava_callback - ERROR - Error processing callback for node 69c3880518a3717aa65d3aea: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43392 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-27 07:01:16,365 - lava_callback - INFO - Received callback for node 69c3882918a3717aa65d3af7 INFO: 172.20.0.1:43406 - "POST /node/69c3882918a3717aa65d3af7 HTTP/1.0" 202 Accepted 2026-03-27 07:01:16,371 - lava_callback - INFO - Starting async job submit for node 69c3882918a3717aa65d3af7 2026-03-27 07:01:16,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:01:16,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3882918a3717aa65d3af7 HTTP/1.1" 200 None 2026-03-27 07:01:16,386 - lava_callback - DEBUG - Processing node 69c3882918a3717aa65d3af7: baseline-arm-clabbe 2026-03-27 07:01:16,387 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3882918a3717aa65d3af7 2026-03-27 07:01:16,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:01:16,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3882918a3717aa65d3af7 HTTP/1.1" 200 None 2026-03-27 07:01:16,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:01:16,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-27 07:01:16,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:01:16,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3882918a3717aa65d3af7 HTTP/1.1" 200 None 2026-03-27 07:01:16,438 - lava_callback - ERROR - Error processing callback for node 69c3882918a3717aa65d3af7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52662 - "GET / HTTP/1.0" 200 OK 2026-03-27 07:01:47,329 - lava_callback - INFO - Received callback for node 69c5d57a2cb62322ad96629e INFO: 172.20.0.1:52670 - "POST /node/69c5d57a2cb62322ad96629e HTTP/1.0" 202 Accepted 2026-03-27 07:01:47,335 - lava_callback - INFO - Starting async job submit for node 69c5d57a2cb62322ad96629e 2026-03-27 07:01:47,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:01:47,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d57a2cb62322ad96629e HTTP/1.1" 200 None 2026-03-27 07:01:47,376 - lava_callback - DEBUG - Processing node 69c5d57a2cb62322ad96629e: baseline-nfs-arm64 2026-03-27 07:01:47,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 07:01:48,168 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 07:01:48,169 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5d57a2cb62322ad96629e/log.txt.gz 2026-03-27 07:01:48,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 07:01:48,636 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 07:01:48,638 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5d57a2cb62322ad96629e/lava-logs.yaml 2026-03-27 07:01:48,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 07:01:48,825 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 07:01:48,827 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5d57a2cb62322ad96629e/lava_callback.json.gz 2026-03-27 07:01:48,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:01:48,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d57a2cb62322ad96629e HTTP/1.1" 200 None 2026-03-27 07:01:48,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:01:48,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-27 07:01:48,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:01:48,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d57a2cb62322ad96629e HTTP/1.1" 200 None 2026-03-27 07:01:48,981 - lava_callback - INFO - Completed processing callback for node 69c5d57a2cb62322ad96629e 2026-03-27 07:01:56,327 - lava_callback - INFO - Received callback for node 69c3884e18a3717aa65d3b04 INFO: 172.20.0.1:34040 - "POST /node/69c3884e18a3717aa65d3b04 HTTP/1.0" 202 Accepted 2026-03-27 07:01:56,332 - lava_callback - INFO - Starting async job submit for node 69c3884e18a3717aa65d3b04 2026-03-27 07:01:56,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:01:56,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3884e18a3717aa65d3b04 HTTP/1.1" 200 None 2026-03-27 07:01:56,348 - lava_callback - DEBUG - Processing node 69c3884e18a3717aa65d3b04: baseline-arm-clabbe 2026-03-27 07:01:56,348 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3884e18a3717aa65d3b04 2026-03-27 07:01:56,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:01:56,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3884e18a3717aa65d3b04 HTTP/1.1" 200 None 2026-03-27 07:01:56,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:01:56,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-27 07:01:56,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:01:56,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3884e18a3717aa65d3b04 HTTP/1.1" 200 None 2026-03-27 07:01:56,401 - lava_callback - ERROR - Error processing callback for node 69c3884e18a3717aa65d3b04: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34044 - "GET / HTTP/1.0" 200 OK 2026-03-27 07:02:01,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:02:01,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:56568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40412 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-27 07:02:16,396 - lava_callback - INFO - Received callback for node 69c3887218a3717aa65d3b10 INFO: 172.20.0.1:40422 - "POST /node/69c3887218a3717aa65d3b10 HTTP/1.0" 202 Accepted 2026-03-27 07:02:16,402 - lava_callback - INFO - Starting async job submit for node 69c3887218a3717aa65d3b10 2026-03-27 07:02:16,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:02:16,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3887218a3717aa65d3b10 HTTP/1.1" 200 None 2026-03-27 07:02:16,422 - lava_callback - DEBUG - Processing node 69c3887218a3717aa65d3b10: baseline-arm-clabbe 2026-03-27 07:02:16,423 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3887218a3717aa65d3b10 2026-03-27 07:02:16,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:02:16,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3887218a3717aa65d3b10 HTTP/1.1" 200 None 2026-03-27 07:02:16,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:02:16,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-27 07:02:16,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:02:16,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3887218a3717aa65d3b10 HTTP/1.1" 200 None 2026-03-27 07:02:16,476 - lava_callback - ERROR - Error processing callback for node 69c3887218a3717aa65d3b10: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38922 - "GET / HTTP/1.0" 200 OK 2026-03-27 07:02:56,370 - lava_callback - INFO - Received callback for node 69c3889618a3717aa65d3b1c INFO: 172.20.0.1:54034 - "POST /node/69c3889618a3717aa65d3b1c HTTP/1.0" 202 Accepted 2026-03-27 07:02:56,379 - lava_callback - INFO - Starting async job submit for node 69c3889618a3717aa65d3b1c 2026-03-27 07:02:56,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:02:56,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3889618a3717aa65d3b1c HTTP/1.1" 200 None 2026-03-27 07:02:56,397 - lava_callback - DEBUG - Processing node 69c3889618a3717aa65d3b1c: baseline-arm-clabbe 2026-03-27 07:02:56,397 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3889618a3717aa65d3b1c 2026-03-27 07:02:56,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:02:56,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3889618a3717aa65d3b1c HTTP/1.1" 200 None 2026-03-27 07:02:56,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:02:56,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-27 07:02:56,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:02:56,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3889618a3717aa65d3b1c HTTP/1.1" 200 None 2026-03-27 07:02:56,457 - lava_callback - ERROR - Error processing callback for node 69c3889618a3717aa65d3b1c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45620 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41272 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41982 - "GET / HTTP/1.0" 200 OK 2026-03-27 07:04:36,419 - lava_callback - INFO - Received callback for node 69c388f918a3717aa65d3b36 INFO: 172.20.0.1:57020 - "POST /node/69c388f918a3717aa65d3b36 HTTP/1.0" 202 Accepted 2026-03-27 07:04:36,424 - lava_callback - INFO - Starting async job submit for node 69c388f918a3717aa65d3b36 2026-03-27 07:04:36,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:04:36,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c388f918a3717aa65d3b36 HTTP/1.1" 200 None 2026-03-27 07:04:36,456 - lava_callback - DEBUG - Processing node 69c388f918a3717aa65d3b36: baseline-arm-clabbe 2026-03-27 07:04:36,456 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c388f918a3717aa65d3b36 2026-03-27 07:04:36,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:04:36,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c388f918a3717aa65d3b36 HTTP/1.1" 200 None 2026-03-27 07:04:36,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:04:36,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-27 07:04:36,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:04:36,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c388f918a3717aa65d3b36 HTTP/1.1" 200 None 2026-03-27 07:04:36,508 - lava_callback - ERROR - Error processing callback for node 69c388f918a3717aa65d3b36: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42934 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-27 07:05:16,463 - lava_callback - INFO - Received callback for node 69c3891f18a3717aa65d3b3e INFO: 172.20.0.1:42948 - "POST /node/69c3891f18a3717aa65d3b3e HTTP/1.0" 202 Accepted 2026-03-27 07:05:16,468 - lava_callback - INFO - Starting async job submit for node 69c3891f18a3717aa65d3b3e 2026-03-27 07:05:16,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:05:16,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3891f18a3717aa65d3b3e HTTP/1.1" 200 None 2026-03-27 07:05:16,485 - lava_callback - DEBUG - Processing node 69c3891f18a3717aa65d3b3e: baseline-arm-clabbe 2026-03-27 07:05:16,485 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3891f18a3717aa65d3b3e 2026-03-27 07:05:16,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:05:16,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3891f18a3717aa65d3b3e HTTP/1.1" 200 None 2026-03-27 07:05:16,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:05:16,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-27 07:05:16,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:05:16,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3891f18a3717aa65d3b3e HTTP/1.1" 200 None 2026-03-27 07:05:16,548 - lava_callback - ERROR - Error processing callback for node 69c3891f18a3717aa65d3b3e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35856 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46408 - "GET / HTTP/1.0" 200 OK 2026-03-27 07:06:56,535 - lava_callback - INFO - Received callback for node 69c3898418a3717aa65d3b54 INFO: 172.20.0.1:43646 - "POST /node/69c3898418a3717aa65d3b54 HTTP/1.0" 202 Accepted 2026-03-27 07:06:56,540 - lava_callback - INFO - Starting async job submit for node 69c3898418a3717aa65d3b54 2026-03-27 07:06:56,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:43658 - "GET / HTTP/1.0" 200 OK 2026-03-27 07:06:56,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3898418a3717aa65d3b54 HTTP/1.1" 200 None 2026-03-27 07:06:56,557 - lava_callback - DEBUG - Processing node 69c3898418a3717aa65d3b54: baseline-arm-clabbe 2026-03-27 07:06:56,557 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3898418a3717aa65d3b54 2026-03-27 07:06:56,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:06:56,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3898418a3717aa65d3b54 HTTP/1.1" 200 None 2026-03-27 07:06:56,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:06:56,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-27 07:06:56,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:06:56,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3898418a3717aa65d3b54 HTTP/1.1" 200 None 2026-03-27 07:06:56,609 - lava_callback - ERROR - Error processing callback for node 69c3898418a3717aa65d3b54: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45248 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58458 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59312 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37878 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36214 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57770 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34124 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49216 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-27 07:14:15,721 - lava_callback - INFO - Received callback for node 69c5e6a92cb62322ad968e93 INFO: 172.20.0.1:49228 - "POST /node/69c5e6a92cb62322ad968e93 HTTP/1.0" 202 Accepted 2026-03-27 07:14:15,728 - lava_callback - INFO - Starting async job submit for node 69c5e6a92cb62322ad968e93 2026-03-27 07:14:15,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:14:15,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5e6a92cb62322ad968e93 HTTP/1.1" 200 None 2026-03-27 07:14:15,768 - lava_callback - DEBUG - Processing node 69c5e6a92cb62322ad968e93: baseline-arm64 2026-03-27 07:14:16,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 07:14:16,451 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 07:14:16,452 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5e6a92cb62322ad968e93/log.txt.gz INFO: 172.20.0.1:49244 - "GET / HTTP/1.0" 200 OK 2026-03-27 07:14:16,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 07:14:16,957 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 07:14:16,959 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5e6a92cb62322ad968e93/lava-logs.yaml 2026-03-27 07:14:16,971 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 07:14:17,144 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 07:14:17,146 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5e6a92cb62322ad968e93/lava_callback.json.gz 2026-03-27 07:14:17,171 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:14:17,185 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5e6a92cb62322ad968e93 HTTP/1.1" 200 None 2026-03-27 07:14:17,187 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:14:17,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-27 07:14:17,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:14:17,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5e6a92cb62322ad968e93 HTTP/1.1" 200 None 2026-03-27 07:14:17,296 - lava_callback - INFO - Completed processing callback for node 69c5e6a92cb62322ad968e93 INFO: 172.20.0.1:49252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60682 - "GET / HTTP/1.0" 200 OK 2026-03-27 07:14:31,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:14:31,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:34428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58974 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34094 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48548 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.20.0.1:40538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33778 - "GET /.well-known/security.txt HTTP/1.0" 404 Not Found INFO: 172.20.0.1:33782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38166 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42276 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36618 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38942 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40986 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51434 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53758 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45480 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44332 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46694 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33814 - "GET / HTTP/1.0" 200 OK 2026-03-27 07:26:29,315 - lava_callback - INFO - Received callback for node 69c38e1918a3717aa65d3bff INFO: 172.20.0.1:33816 - "POST /node/69c38e1918a3717aa65d3bff HTTP/1.0" 202 Accepted 2026-03-27 07:26:29,321 - lava_callback - INFO - Starting async job submit for node 69c38e1918a3717aa65d3bff 2026-03-27 07:26:29,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:26:29,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c38e1918a3717aa65d3bff HTTP/1.1" 200 None 2026-03-27 07:26:29,339 - lava_callback - DEBUG - Processing node 69c38e1918a3717aa65d3bff: baseline-arm64-baylibre 2026-03-27 07:26:29,339 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c38e1918a3717aa65d3bff 2026-03-27 07:26:29,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:26:29,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c38e1918a3717aa65d3bff HTTP/1.1" 200 None 2026-03-27 07:26:29,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:26:29,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-27 07:26:29,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:26:29,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c38e1918a3717aa65d3bff HTTP/1.1" 200 None 2026-03-27 07:26:29,396 - lava_callback - ERROR - Error processing callback for node 69c38e1918a3717aa65d3bff: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35178 - "GET / HTTP/1.0" 200 OK 2026-03-27 07:26:55,178 - lava_callback - INFO - Received callback for node 69c5e8082cb62322ad968ef0 INFO: 172.20.0.1:52160 - "POST /node/69c5e8082cb62322ad968ef0 HTTP/1.0" 202 Accepted 2026-03-27 07:26:55,185 - lava_callback - INFO - Starting async job submit for node 69c5e8082cb62322ad968ef0 2026-03-27 07:26:55,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:26:55,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5e8082cb62322ad968ef0 HTTP/1.1" 200 None 2026-03-27 07:26:55,228 - lava_callback - DEBUG - Processing node 69c5e8082cb62322ad968ef0: baseline-arm64 2026-03-27 07:26:55,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 07:26:55,908 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 07:26:55,909 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5e8082cb62322ad968ef0/log.txt.gz 2026-03-27 07:26:56,163 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 07:26:56,436 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 07:26:56,438 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5e8082cb62322ad968ef0/lava-logs.yaml 2026-03-27 07:26:56,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:52164 - "GET / HTTP/1.0" 200 OK 2026-03-27 07:26:56,635 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 07:26:56,637 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5e8082cb62322ad968ef0/lava_callback.json.gz 2026-03-27 07:26:56,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:26:56,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5e8082cb62322ad968ef0 HTTP/1.1" 200 None 2026-03-27 07:26:56,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:26:56,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-27 07:26:56,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:26:56,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5e8082cb62322ad968ef0 HTTP/1.1" 200 None 2026-03-27 07:26:56,792 - lava_callback - INFO - Completed processing callback for node 69c5e8082cb62322ad968ef0 2026-03-27 07:27:01,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:27:01,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:47288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48286 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47188 - "GET / HTTP/1.0" 200 OK 2026-03-27 07:27:49,393 - lava_callback - INFO - Received callback for node 69c38e6618a3717aa65d3c0b INFO: 172.20.0.1:47198 - "POST /node/69c38e6618a3717aa65d3c0b HTTP/1.0" 202 Accepted 2026-03-27 07:27:49,399 - lava_callback - INFO - Starting async job submit for node 69c38e6618a3717aa65d3c0b 2026-03-27 07:27:49,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:27:49,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c38e6618a3717aa65d3c0b HTTP/1.1" 200 None 2026-03-27 07:27:49,415 - lava_callback - DEBUG - Processing node 69c38e6618a3717aa65d3c0b: baseline-arm64-baylibre 2026-03-27 07:27:49,415 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c38e6618a3717aa65d3c0b 2026-03-27 07:27:49,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:27:49,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c38e6618a3717aa65d3c0b HTTP/1.1" 200 None 2026-03-27 07:27:49,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:27:49,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-27 07:27:49,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:27:49,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c38e6618a3717aa65d3c0b HTTP/1.1" 200 None 2026-03-27 07:27:49,484 - lava_callback - ERROR - Error processing callback for node 69c38e6618a3717aa65d3c0b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47208 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41932 - "GET / HTTP/1.0" 200 OK 2026-03-27 07:29:09,508 - lava_callback - INFO - Received callback for node 69c38eb318a3717aa65d3c10 INFO: 172.20.0.1:41948 - "POST /node/69c38eb318a3717aa65d3c10 HTTP/1.0" 202 Accepted 2026-03-27 07:29:09,514 - lava_callback - INFO - Starting async job submit for node 69c38eb318a3717aa65d3c10 2026-03-27 07:29:09,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:29:09,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c38eb318a3717aa65d3c10 HTTP/1.1" 200 None 2026-03-27 07:29:09,530 - lava_callback - DEBUG - Processing node 69c38eb318a3717aa65d3c10: baseline-arm64-baylibre 2026-03-27 07:29:09,531 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c38eb318a3717aa65d3c10 2026-03-27 07:29:09,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:29:09,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c38eb318a3717aa65d3c10 HTTP/1.1" 200 None 2026-03-27 07:29:09,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:29:09,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-27 07:29:09,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:29:09,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c38eb318a3717aa65d3c10 HTTP/1.1" 200 None 2026-03-27 07:29:09,585 - lava_callback - ERROR - Error processing callback for node 69c38eb318a3717aa65d3c10: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58136 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57668 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37282 - "GET / HTTP/1.0" 200 OK 2026-03-27 07:30:29,520 - lava_callback - INFO - Received callback for node 69c38f0318a3717aa65d3c1c INFO: 172.20.0.1:37284 - "POST /node/69c38f0318a3717aa65d3c1c HTTP/1.0" 202 Accepted 2026-03-27 07:30:29,526 - lava_callback - INFO - Starting async job submit for node 69c38f0318a3717aa65d3c1c 2026-03-27 07:30:29,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:30:29,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c38f0318a3717aa65d3c1c HTTP/1.1" 200 None 2026-03-27 07:30:29,542 - lava_callback - DEBUG - Processing node 69c38f0318a3717aa65d3c1c: baseline-arm64-baylibre 2026-03-27 07:30:29,542 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c38f0318a3717aa65d3c1c 2026-03-27 07:30:29,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:30:29,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c38f0318a3717aa65d3c1c HTTP/1.1" 200 None 2026-03-27 07:30:29,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:30:29,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-27 07:30:29,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:30:29,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c38f0318a3717aa65d3c1c HTTP/1.1" 200 None 2026-03-27 07:30:29,601 - lava_callback - ERROR - Error processing callback for node 69c38f0318a3717aa65d3c1c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37790 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53834 - "GET / HTTP/1.0" 200 OK 2026-03-27 07:31:49,621 - lava_callback - INFO - Received callback for node 69c38f5118a3717aa65d3c28 INFO: 172.20.0.1:53842 - "POST /node/69c38f5118a3717aa65d3c28 HTTP/1.0" 202 Accepted 2026-03-27 07:31:49,626 - lava_callback - INFO - Starting async job submit for node 69c38f5118a3717aa65d3c28 2026-03-27 07:31:49,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:31:49,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c38f5118a3717aa65d3c28 HTTP/1.1" 200 None 2026-03-27 07:31:49,643 - lava_callback - DEBUG - Processing node 69c38f5118a3717aa65d3c28: baseline-arm64-baylibre 2026-03-27 07:31:49,643 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c38f5118a3717aa65d3c28 2026-03-27 07:31:49,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:31:49,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c38f5118a3717aa65d3c28 HTTP/1.1" 200 None 2026-03-27 07:31:49,659 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:31:49,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-27 07:31:49,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:31:49,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c38f5118a3717aa65d3c28 HTTP/1.1" 200 None 2026-03-27 07:31:49,720 - lava_callback - ERROR - Error processing callback for node 69c38f5118a3717aa65d3c28: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40082 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59548 - "GET / HTTP/1.0" 200 OK 2026-03-27 07:32:49,599 - lava_callback - INFO - Received callback for node 69c38f9e18a3717aa65d3c2f INFO: 172.20.0.1:59560 - "POST /node/69c38f9e18a3717aa65d3c2f HTTP/1.0" 202 Accepted 2026-03-27 07:32:49,605 - lava_callback - INFO - Starting async job submit for node 69c38f9e18a3717aa65d3c2f 2026-03-27 07:32:49,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:32:49,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c38f9e18a3717aa65d3c2f HTTP/1.1" 200 None 2026-03-27 07:32:49,628 - lava_callback - DEBUG - Processing node 69c38f9e18a3717aa65d3c2f: baseline-arm64-baylibre 2026-03-27 07:32:49,628 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c38f9e18a3717aa65d3c2f 2026-03-27 07:32:49,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:32:49,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c38f9e18a3717aa65d3c2f HTTP/1.1" 200 None 2026-03-27 07:32:49,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:32:49,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-27 07:32:49,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:32:49,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c38f9e18a3717aa65d3c2f HTTP/1.1" 200 None 2026-03-27 07:32:49,690 - lava_callback - ERROR - Error processing callback for node 69c38f9e18a3717aa65d3c2f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36702 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44262 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50796 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48930 - "GET / HTTP/1.0" 200 OK 2026-03-27 07:35:37,689 - lava_callback - INFO - Received callback for node 69c3903d18a3717aa65d3c43 INFO: 172.20.0.1:48946 - "POST /node/69c3903d18a3717aa65d3c43 HTTP/1.0" 202 Accepted 2026-03-27 07:35:37,695 - lava_callback - INFO - Starting async job submit for node 69c3903d18a3717aa65d3c43 2026-03-27 07:35:37,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:35:37,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3903d18a3717aa65d3c43 HTTP/1.1" 200 None 2026-03-27 07:35:37,711 - lava_callback - DEBUG - Processing node 69c3903d18a3717aa65d3c43: baseline-arm64-clabbe 2026-03-27 07:35:37,711 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3903d18a3717aa65d3c43 2026-03-27 07:35:37,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:35:37,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3903d18a3717aa65d3c43 HTTP/1.1" 200 None 2026-03-27 07:35:37,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:35:37,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-27 07:35:37,743 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:35:37,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3903d18a3717aa65d3c43 HTTP/1.1" 200 None 2026-03-27 07:35:37,770 - lava_callback - ERROR - Error processing callback for node 69c3903d18a3717aa65d3c43: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57538 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36314 - "GET / HTTP/1.0" 200 OK 2026-03-27 07:36:57,683 - lava_callback - INFO - Received callback for node 69c3908b18a3717aa65d3c4e INFO: 172.20.0.1:36326 - "POST /node/69c3908b18a3717aa65d3c4e HTTP/1.0" 202 Accepted 2026-03-27 07:36:57,689 - lava_callback - INFO - Starting async job submit for node 69c3908b18a3717aa65d3c4e 2026-03-27 07:36:57,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:36:57,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3908b18a3717aa65d3c4e HTTP/1.1" 200 None 2026-03-27 07:36:57,731 - lava_callback - DEBUG - Processing node 69c3908b18a3717aa65d3c4e: baseline-arm64-clabbe 2026-03-27 07:36:57,731 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3908b18a3717aa65d3c4e 2026-03-27 07:36:57,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:36:57,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3908b18a3717aa65d3c4e HTTP/1.1" 200 None 2026-03-27 07:36:57,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:36:57,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-27 07:36:57,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:36:57,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3908b18a3717aa65d3c4e HTTP/1.1" 200 None 2026-03-27 07:36:57,787 - lava_callback - ERROR - Error processing callback for node 69c3908b18a3717aa65d3c4e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60270 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41424 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41428 - "GET / HTTP/1.0" 200 OK 2026-03-27 07:38:17,750 - lava_callback - INFO - Received callback for node 69c390d818a3717aa65d3c58 INFO: 172.20.0.1:41430 - "POST /node/69c390d818a3717aa65d3c58 HTTP/1.0" 202 Accepted 2026-03-27 07:38:17,755 - lava_callback - INFO - Starting async job submit for node 69c390d818a3717aa65d3c58 2026-03-27 07:38:17,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:38:17,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c390d818a3717aa65d3c58 HTTP/1.1" 200 None 2026-03-27 07:38:17,781 - lava_callback - DEBUG - Processing node 69c390d818a3717aa65d3c58: baseline-arm64-clabbe 2026-03-27 07:38:17,782 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c390d818a3717aa65d3c58 2026-03-27 07:38:17,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:38:17,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c390d818a3717aa65d3c58 HTTP/1.1" 200 None 2026-03-27 07:38:17,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:38:17,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-27 07:38:17,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:38:17,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c390d818a3717aa65d3c58 HTTP/1.1" 200 None 2026-03-27 07:38:17,863 - lava_callback - ERROR - Error processing callback for node 69c390d818a3717aa65d3c58: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48880 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47008 - "GET / HTTP/1.0" 200 OK 2026-03-27 07:39:37,773 - lava_callback - INFO - Received callback for node 69c3912518a3717aa65d3c61 INFO: 172.20.0.1:47022 - "POST /node/69c3912518a3717aa65d3c61 HTTP/1.0" 202 Accepted 2026-03-27 07:39:37,779 - lava_callback - INFO - Starting async job submit for node 69c3912518a3717aa65d3c61 2026-03-27 07:39:37,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:39:37,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3912518a3717aa65d3c61 HTTP/1.1" 200 None 2026-03-27 07:39:37,800 - lava_callback - DEBUG - Processing node 69c3912518a3717aa65d3c61: baseline-arm64-clabbe 2026-03-27 07:39:37,800 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3912518a3717aa65d3c61 2026-03-27 07:39:37,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:39:37,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3912518a3717aa65d3c61 HTTP/1.1" 200 None 2026-03-27 07:39:37,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:39:37,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-27 07:39:37,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:39:37,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3912518a3717aa65d3c61 HTTP/1.1" 200 None 2026-03-27 07:39:37,857 - lava_callback - ERROR - Error processing callback for node 69c3912518a3717aa65d3c61: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:32834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45718 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33906 - "GET / HTTP/1.0" 200 OK 2026-03-27 07:40:57,813 - lava_callback - INFO - Received callback for node 69c3917318a3717aa65d3c6c INFO: 172.20.0.1:33916 - "POST /node/69c3917318a3717aa65d3c6c HTTP/1.0" 202 Accepted 2026-03-27 07:40:57,819 - lava_callback - INFO - Starting async job submit for node 69c3917318a3717aa65d3c6c 2026-03-27 07:40:57,819 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:40:57,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3917318a3717aa65d3c6c HTTP/1.1" 200 None 2026-03-27 07:40:57,835 - lava_callback - DEBUG - Processing node 69c3917318a3717aa65d3c6c: baseline-arm64-clabbe 2026-03-27 07:40:57,835 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3917318a3717aa65d3c6c 2026-03-27 07:40:57,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:40:57,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3917318a3717aa65d3c6c HTTP/1.1" 200 None 2026-03-27 07:40:57,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:40:57,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-27 07:40:57,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:40:57,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3917318a3717aa65d3c6c HTTP/1.1" 200 None 2026-03-27 07:40:57,917 - lava_callback - ERROR - Error processing callback for node 69c3917318a3717aa65d3c6c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34846 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46598 - "GET / HTTP/1.0" 200 OK 2026-03-27 07:41:57,822 - lava_callback - INFO - Received callback for node 69c391c218a3717aa65d3c74 INFO: 172.20.0.1:46612 - "POST /node/69c391c218a3717aa65d3c74 HTTP/1.0" 202 Accepted 2026-03-27 07:41:57,828 - lava_callback - INFO - Starting async job submit for node 69c391c218a3717aa65d3c74 2026-03-27 07:41:57,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:41:57,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c391c218a3717aa65d3c74 HTTP/1.1" 200 None 2026-03-27 07:41:57,845 - lava_callback - DEBUG - Processing node 69c391c218a3717aa65d3c74: baseline-arm64-clabbe 2026-03-27 07:41:57,845 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c391c218a3717aa65d3c74 2026-03-27 07:41:57,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:41:57,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c391c218a3717aa65d3c74 HTTP/1.1" 200 None 2026-03-27 07:41:57,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:41:57,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-27 07:41:57,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:41:57,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c391c218a3717aa65d3c74 HTTP/1.1" 200 None 2026-03-27 07:41:57,901 - lava_callback - ERROR - Error processing callback for node 69c391c218a3717aa65d3c74: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53038 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50436 - "GET / HTTP/1.0" 200 OK 2026-03-27 07:43:10,540 - lava_callback - INFO - Received callback for node 69c3920b18a3717aa65d3c81 INFO: 172.20.0.1:60144 - "POST /node/69c3920b18a3717aa65d3c81 HTTP/1.0" 202 Accepted 2026-03-27 07:43:10,546 - lava_callback - INFO - Starting async job submit for node 69c3920b18a3717aa65d3c81 2026-03-27 07:43:10,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:43:10,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3920b18a3717aa65d3c81 HTTP/1.1" 200 None 2026-03-27 07:43:10,565 - lava_callback - DEBUG - Processing node 69c3920b18a3717aa65d3c81: baseline-arm64-baylibre 2026-03-27 07:43:10,565 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3920b18a3717aa65d3c81 2026-03-27 07:43:10,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:43:10,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3920b18a3717aa65d3c81 HTTP/1.1" 200 None 2026-03-27 07:43:10,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:43:10,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-27 07:43:10,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:43:10,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3920b18a3717aa65d3c81 HTTP/1.1" 200 None 2026-03-27 07:43:10,622 - lava_callback - ERROR - Error processing callback for node 69c3920b18a3717aa65d3c81: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60158 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52754 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33926 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56488 - "GET / HTTP/1.0" 200 OK 2026-03-27 07:45:30,423 - lava_callback - INFO - Received callback for node 69c3929318a3717aa65d3c8e INFO: 172.20.0.1:51728 - "POST /node/69c3929318a3717aa65d3c8e HTTP/1.0" 202 Accepted 2026-03-27 07:45:30,429 - lava_callback - INFO - Starting async job submit for node 69c3929318a3717aa65d3c8e 2026-03-27 07:45:30,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:45:30,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3929318a3717aa65d3c8e HTTP/1.1" 200 None 2026-03-27 07:45:30,445 - lava_callback - DEBUG - Processing node 69c3929318a3717aa65d3c8e: baseline-arm64-baylibre 2026-03-27 07:45:30,445 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3929318a3717aa65d3c8e 2026-03-27 07:45:30,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:45:30,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3929318a3717aa65d3c8e HTTP/1.1" 200 None 2026-03-27 07:45:30,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:45:30,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-27 07:45:30,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:45:30,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3929318a3717aa65d3c8e HTTP/1.1" 200 None 2026-03-27 07:45:30,506 - lava_callback - ERROR - Error processing callback for node 69c3929318a3717aa65d3c8e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39470 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35276 - "GET / HTTP/1.0" 200 OK 2026-03-27 07:46:50,439 - lava_callback - INFO - Received callback for node 69c392e718a3717aa65d3c97 INFO: 172.20.0.1:51566 - "POST /node/69c392e718a3717aa65d3c97 HTTP/1.0" 202 Accepted 2026-03-27 07:46:50,444 - lava_callback - INFO - Starting async job submit for node 69c392e718a3717aa65d3c97 2026-03-27 07:46:50,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:46:50,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c392e718a3717aa65d3c97 HTTP/1.1" 200 None 2026-03-27 07:46:50,460 - lava_callback - DEBUG - Processing node 69c392e718a3717aa65d3c97: baseline-arm64-baylibre 2026-03-27 07:46:50,461 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c392e718a3717aa65d3c97 2026-03-27 07:46:50,461 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:46:50,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c392e718a3717aa65d3c97 HTTP/1.1" 200 None 2026-03-27 07:46:50,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:46:50,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-27 07:46:50,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:46:50,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c392e718a3717aa65d3c97 HTTP/1.1" 200 None 2026-03-27 07:46:50,518 - lava_callback - ERROR - Error processing callback for node 69c392e718a3717aa65d3c97: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38654 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48172 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60626 - "GET / HTTP/1.0" 200 OK 2026-03-27 07:48:30,689 - lava_callback - INFO - Received callback for node 69c3933a18a3717aa65d3ca4 INFO: 172.20.0.1:53262 - "POST /node/69c3933a18a3717aa65d3ca4 HTTP/1.0" 202 Accepted 2026-03-27 07:48:30,695 - lava_callback - INFO - Starting async job submit for node 69c3933a18a3717aa65d3ca4 2026-03-27 07:48:30,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:48:30,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3933a18a3717aa65d3ca4 HTTP/1.1" 200 None 2026-03-27 07:48:30,711 - lava_callback - DEBUG - Processing node 69c3933a18a3717aa65d3ca4: baseline-arm64-baylibre 2026-03-27 07:48:30,711 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3933a18a3717aa65d3ca4 2026-03-27 07:48:30,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:48:30,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3933a18a3717aa65d3ca4 HTTP/1.1" 200 None 2026-03-27 07:48:30,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:48:30,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-27 07:48:30,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:48:30,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3933a18a3717aa65d3ca4 HTTP/1.1" 200 None 2026-03-27 07:48:30,792 - lava_callback - ERROR - Error processing callback for node 69c3933a18a3717aa65d3ca4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57296 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35550 - "GET / HTTP/1.0" 200 OK 2026-03-27 07:49:30,528 - lava_callback - INFO - Received callback for node 69c3938318a3717aa65d3cac INFO: 172.20.0.1:56774 - "POST /node/69c3938318a3717aa65d3cac HTTP/1.0" 202 Accepted 2026-03-27 07:49:30,539 - lava_callback - INFO - Starting async job submit for node 69c3938318a3717aa65d3cac 2026-03-27 07:49:30,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:49:30,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3938318a3717aa65d3cac HTTP/1.1" 200 None 2026-03-27 07:49:30,580 - lava_callback - DEBUG - Processing node 69c3938318a3717aa65d3cac: baseline-arm64-baylibre 2026-03-27 07:49:30,580 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3938318a3717aa65d3cac 2026-03-27 07:49:30,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:49:30,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3938318a3717aa65d3cac HTTP/1.1" 200 None 2026-03-27 07:49:30,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:49:30,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-27 07:49:30,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:49:30,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3938318a3717aa65d3cac HTTP/1.1" 200 None 2026-03-27 07:49:30,640 - lava_callback - ERROR - Error processing callback for node 69c3938318a3717aa65d3cac: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44450 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36616 - "GET / HTTP/1.0" 200 OK 2026-03-27 07:50:50,685 - lava_callback - INFO - Received callback for node 69c393d118a3717aa65d3cbb INFO: 172.20.0.1:56794 - "POST /node/69c393d118a3717aa65d3cbb HTTP/1.0" 202 Accepted 2026-03-27 07:50:50,691 - lava_callback - INFO - Starting async job submit for node 69c393d118a3717aa65d3cbb 2026-03-27 07:50:50,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:50:50,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c393d118a3717aa65d3cbb HTTP/1.1" 200 None 2026-03-27 07:50:50,707 - lava_callback - DEBUG - Processing node 69c393d118a3717aa65d3cbb: baseline-arm64-baylibre 2026-03-27 07:50:50,708 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c393d118a3717aa65d3cbb 2026-03-27 07:50:50,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:50:50,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c393d118a3717aa65d3cbb HTTP/1.1" 200 None 2026-03-27 07:50:50,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:50:50,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-27 07:50:50,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:50:50,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c393d118a3717aa65d3cbb HTTP/1.1" 200 None 2026-03-27 07:50:50,758 - lava_callback - ERROR - Error processing callback for node 69c393d118a3717aa65d3cbb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42206 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43604 - "GET / HTTP/1.0" 200 OK 2026-03-27 07:52:10,722 - lava_callback - INFO - Received callback for node 69c3941a18a3717aa65d3cc1 INFO: 172.20.0.1:33710 - "POST /node/69c3941a18a3717aa65d3cc1 HTTP/1.0" 202 Accepted 2026-03-27 07:52:10,728 - lava_callback - INFO - Starting async job submit for node 69c3941a18a3717aa65d3cc1 2026-03-27 07:52:10,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:52:10,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3941a18a3717aa65d3cc1 HTTP/1.1" 200 None 2026-03-27 07:52:10,747 - lava_callback - DEBUG - Processing node 69c3941a18a3717aa65d3cc1: baseline-arm64-baylibre 2026-03-27 07:52:10,747 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3941a18a3717aa65d3cc1 2026-03-27 07:52:10,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:52:10,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3941a18a3717aa65d3cc1 HTTP/1.1" 200 None 2026-03-27 07:52:10,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:52:10,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-27 07:52:10,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:52:10,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3941a18a3717aa65d3cc1 HTTP/1.1" 200 None 2026-03-27 07:52:10,800 - lava_callback - ERROR - Error processing callback for node 69c3941a18a3717aa65d3cc1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33720 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40922 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40040 - "GET / HTTP/1.0" 200 OK 2026-03-27 07:53:30,761 - lava_callback - INFO - Received callback for node 69c3947318a3717aa65d3ccf INFO: 172.20.0.1:45046 - "POST /node/69c3947318a3717aa65d3ccf HTTP/1.0" 202 Accepted 2026-03-27 07:53:30,767 - lava_callback - INFO - Starting async job submit for node 69c3947318a3717aa65d3ccf 2026-03-27 07:53:30,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:53:30,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3947318a3717aa65d3ccf HTTP/1.1" 200 None 2026-03-27 07:53:30,783 - lava_callback - DEBUG - Processing node 69c3947318a3717aa65d3ccf: baseline-arm64-baylibre 2026-03-27 07:53:30,784 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3947318a3717aa65d3ccf 2026-03-27 07:53:30,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:53:30,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3947318a3717aa65d3ccf HTTP/1.1" 200 None 2026-03-27 07:53:30,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:53:30,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-27 07:53:30,819 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:53:30,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3947318a3717aa65d3ccf HTTP/1.1" 200 None 2026-03-27 07:53:30,847 - lava_callback - ERROR - Error processing callback for node 69c3947318a3717aa65d3ccf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33844 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36274 - "GET / HTTP/1.0" 200 OK 2026-03-27 07:54:50,865 - lava_callback - INFO - Received callback for node 69c394c218a3717aa65d3cd9 INFO: 172.20.0.1:59496 - "POST /node/69c394c218a3717aa65d3cd9 HTTP/1.0" 202 Accepted 2026-03-27 07:54:50,871 - lava_callback - INFO - Starting async job submit for node 69c394c218a3717aa65d3cd9 2026-03-27 07:54:50,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:54:50,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c394c218a3717aa65d3cd9 HTTP/1.1" 200 None 2026-03-27 07:54:50,899 - lava_callback - DEBUG - Processing node 69c394c218a3717aa65d3cd9: baseline-arm64-baylibre 2026-03-27 07:54:50,899 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c394c218a3717aa65d3cd9 2026-03-27 07:54:50,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:54:50,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c394c218a3717aa65d3cd9 HTTP/1.1" 200 None 2026-03-27 07:54:50,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:54:50,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-27 07:54:50,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:54:50,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c394c218a3717aa65d3cd9 HTTP/1.1" 200 None 2026-03-27 07:54:50,958 - lava_callback - ERROR - Error processing callback for node 69c394c218a3717aa65d3cd9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57946 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50958 - "GET / HTTP/1.0" 200 OK 2026-03-27 07:56:10,955 - lava_callback - INFO - Received callback for node 69c3950e18a3717aa65d3ce9 INFO: 172.20.0.1:34274 - "POST /node/69c3950e18a3717aa65d3ce9 HTTP/1.0" 202 Accepted 2026-03-27 07:56:10,961 - lava_callback - INFO - Starting async job submit for node 69c3950e18a3717aa65d3ce9 2026-03-27 07:56:10,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:56:10,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3950e18a3717aa65d3ce9 HTTP/1.1" 200 None 2026-03-27 07:56:10,977 - lava_callback - DEBUG - Processing node 69c3950e18a3717aa65d3ce9: baseline-arm64-baylibre 2026-03-27 07:56:10,977 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3950e18a3717aa65d3ce9 2026-03-27 07:56:10,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:56:10,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3950e18a3717aa65d3ce9 HTTP/1.1" 200 None 2026-03-27 07:56:10,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:56:11,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-27 07:56:11,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:56:11,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3950e18a3717aa65d3ce9 HTTP/1.1" 200 None 2026-03-27 07:56:11,037 - lava_callback - ERROR - Error processing callback for node 69c3950e18a3717aa65d3ce9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34276 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40726 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58696 - "GET / HTTP/1.0" 200 OK 2026-03-27 07:57:31,023 - lava_callback - INFO - Received callback for node 69c3955b18a3717aa65d3cef INFO: 172.20.0.1:40024 - "POST /node/69c3955b18a3717aa65d3cef HTTP/1.0" 202 Accepted 2026-03-27 07:57:31,028 - lava_callback - INFO - Starting async job submit for node 69c3955b18a3717aa65d3cef 2026-03-27 07:57:31,029 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:57:31,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3955b18a3717aa65d3cef HTTP/1.1" 200 None 2026-03-27 07:57:31,054 - lava_callback - DEBUG - Processing node 69c3955b18a3717aa65d3cef: baseline-arm64-baylibre 2026-03-27 07:57:31,054 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3955b18a3717aa65d3cef 2026-03-27 07:57:31,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:57:31,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3955b18a3717aa65d3cef HTTP/1.1" 200 None 2026-03-27 07:57:31,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:57:31,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-27 07:57:31,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:57:31,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3955b18a3717aa65d3cef HTTP/1.1" 200 None 2026-03-27 07:57:31,116 - lava_callback - ERROR - Error processing callback for node 69c3955b18a3717aa65d3cef: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39472 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55952 - "GET / HTTP/1.0" 200 OK 2026-03-27 07:58:51,085 - lava_callback - INFO - Received callback for node 69c395a718a3717aa65d3cff INFO: 172.20.0.1:50380 - "POST /node/69c395a718a3717aa65d3cff HTTP/1.0" 202 Accepted 2026-03-27 07:58:51,091 - lava_callback - INFO - Starting async job submit for node 69c395a718a3717aa65d3cff 2026-03-27 07:58:51,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:58:51,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c395a718a3717aa65d3cff HTTP/1.1" 200 None 2026-03-27 07:58:51,124 - lava_callback - DEBUG - Processing node 69c395a718a3717aa65d3cff: baseline-arm64-baylibre 2026-03-27 07:58:51,125 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c395a718a3717aa65d3cff 2026-03-27 07:58:51,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:58:51,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c395a718a3717aa65d3cff HTTP/1.1" 200 None 2026-03-27 07:58:51,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:58:51,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-27 07:58:51,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:58:51,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c395a718a3717aa65d3cff HTTP/1.1" 200 None 2026-03-27 07:58:51,179 - lava_callback - ERROR - Error processing callback for node 69c395a718a3717aa65d3cff: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52138 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51600 - "GET / HTTP/1.0" 200 OK 2026-03-27 07:59:58,625 - lava_callback - INFO - Received callback for node 69c395f918a3717aa65d3d08 INFO: 172.20.0.1:51616 - "POST /node/69c395f918a3717aa65d3d08 HTTP/1.0" 202 Accepted 2026-03-27 07:59:58,631 - lava_callback - INFO - Starting async job submit for node 69c395f918a3717aa65d3d08 2026-03-27 07:59:58,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:59:58,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c395f918a3717aa65d3d08 HTTP/1.1" 200 None 2026-03-27 07:59:58,646 - lava_callback - DEBUG - Processing node 69c395f918a3717aa65d3d08: baseline-arm64-clabbe 2026-03-27 07:59:58,647 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c395f918a3717aa65d3d08 2026-03-27 07:59:58,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:59:58,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c395f918a3717aa65d3d08 HTTP/1.1" 200 None 2026-03-27 07:59:58,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:59:58,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-27 07:59:58,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 07:59:58,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c395f918a3717aa65d3d08 HTTP/1.1" 200 None 2026-03-27 07:59:58,702 - lava_callback - ERROR - Error processing callback for node 69c395f918a3717aa65d3d08: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36650 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48190 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48204 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:01:18,688 - lava_callback - INFO - Received callback for node 69c3964618a3717aa65d3d14 INFO: 172.20.0.1:48206 - "POST /node/69c3964618a3717aa65d3d14 HTTP/1.0" 202 Accepted 2026-03-27 08:01:18,694 - lava_callback - INFO - Starting async job submit for node 69c3964618a3717aa65d3d14 2026-03-27 08:01:18,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:01:18,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3964618a3717aa65d3d14 HTTP/1.1" 200 None 2026-03-27 08:01:18,710 - lava_callback - DEBUG - Processing node 69c3964618a3717aa65d3d14: baseline-arm64-clabbe 2026-03-27 08:01:18,710 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3964618a3717aa65d3d14 2026-03-27 08:01:18,711 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:01:18,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3964618a3717aa65d3d14 HTTP/1.1" 200 None 2026-03-27 08:01:18,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:01:18,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-27 08:01:18,742 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:01:18,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3964618a3717aa65d3d14 HTTP/1.1" 200 None 2026-03-27 08:01:18,769 - lava_callback - ERROR - Error processing callback for node 69c3964618a3717aa65d3d14: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36904 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43600 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:02:38,777 - lava_callback - INFO - Received callback for node 69c3969218a3717aa65d3d1e INFO: 172.20.0.1:43608 - "POST /node/69c3969218a3717aa65d3d1e HTTP/1.0" 202 Accepted 2026-03-27 08:02:38,782 - lava_callback - INFO - Starting async job submit for node 69c3969218a3717aa65d3d1e 2026-03-27 08:02:38,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:02:38,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3969218a3717aa65d3d1e HTTP/1.1" 200 None 2026-03-27 08:02:38,798 - lava_callback - DEBUG - Processing node 69c3969218a3717aa65d3d1e: baseline-arm64-clabbe 2026-03-27 08:02:38,799 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3969218a3717aa65d3d1e 2026-03-27 08:02:38,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:02:38,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3969218a3717aa65d3d1e HTTP/1.1" 200 None 2026-03-27 08:02:38,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:02:38,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-27 08:02:38,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:02:38,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3969218a3717aa65d3d1e HTTP/1.1" 200 None 2026-03-27 08:02:38,872 - lava_callback - ERROR - Error processing callback for node 69c3969218a3717aa65d3d1e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34766 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40646 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:03:58,855 - lava_callback - INFO - Received callback for node 69c396df18a3717aa65d3d2b INFO: 172.20.0.1:40656 - "POST /node/69c396df18a3717aa65d3d2b HTTP/1.0" 202 Accepted 2026-03-27 08:03:58,861 - lava_callback - INFO - Starting async job submit for node 69c396df18a3717aa65d3d2b 2026-03-27 08:03:58,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:03:58,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c396df18a3717aa65d3d2b HTTP/1.1" 200 None 2026-03-27 08:03:58,877 - lava_callback - DEBUG - Processing node 69c396df18a3717aa65d3d2b: baseline-arm64-clabbe 2026-03-27 08:03:58,877 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c396df18a3717aa65d3d2b 2026-03-27 08:03:58,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:03:58,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c396df18a3717aa65d3d2b HTTP/1.1" 200 None 2026-03-27 08:03:58,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:03:58,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-27 08:03:58,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:03:58,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c396df18a3717aa65d3d2b HTTP/1.1" 200 None 2026-03-27 08:03:58,943 - lava_callback - ERROR - Error processing callback for node 69c396df18a3717aa65d3d2b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60546 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43848 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43860 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:05:18,878 - lava_callback - INFO - Received callback for node 69c3972b18a3717aa65d3d38 INFO: 172.20.0.1:43874 - "POST /node/69c3972b18a3717aa65d3d38 HTTP/1.0" 202 Accepted 2026-03-27 08:05:18,883 - lava_callback - INFO - Starting async job submit for node 69c3972b18a3717aa65d3d38 2026-03-27 08:05:18,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:05:18,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3972b18a3717aa65d3d38 HTTP/1.1" 200 None 2026-03-27 08:05:18,899 - lava_callback - DEBUG - Processing node 69c3972b18a3717aa65d3d38: baseline-arm64-clabbe 2026-03-27 08:05:18,899 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3972b18a3717aa65d3d38 2026-03-27 08:05:18,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:05:18,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3972b18a3717aa65d3d38 HTTP/1.1" 200 None 2026-03-27 08:05:18,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:05:18,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-27 08:05:18,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:05:18,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3972b18a3717aa65d3d38 HTTP/1.1" 200 None 2026-03-27 08:05:18,958 - lava_callback - ERROR - Error processing callback for node 69c3972b18a3717aa65d3d38: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34964 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43404 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:06:39,033 - lava_callback - INFO - Received callback for node 69c3977818a3717aa65d3d45 INFO: 172.20.0.1:43406 - "POST /node/69c3977818a3717aa65d3d45 HTTP/1.0" 202 Accepted 2026-03-27 08:06:39,039 - lava_callback - INFO - Starting async job submit for node 69c3977818a3717aa65d3d45 2026-03-27 08:06:39,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:06:39,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3977818a3717aa65d3d45 HTTP/1.1" 200 None 2026-03-27 08:06:39,055 - lava_callback - DEBUG - Processing node 69c3977818a3717aa65d3d45: baseline-arm64-clabbe 2026-03-27 08:06:39,055 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3977818a3717aa65d3d45 2026-03-27 08:06:39,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:06:39,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3977818a3717aa65d3d45 HTTP/1.1" 200 None 2026-03-27 08:06:39,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:06:39,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-27 08:06:39,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:06:39,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3977818a3717aa65d3d45 HTTP/1.1" 200 None 2026-03-27 08:06:39,124 - lava_callback - ERROR - Error processing callback for node 69c3977818a3717aa65d3d45: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41298 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41166 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:07:39,098 - lava_callback - INFO - Received callback for node 69c397c518a3717aa65d3d51 INFO: 172.20.0.1:41182 - "POST /node/69c397c518a3717aa65d3d51 HTTP/1.0" 202 Accepted 2026-03-27 08:07:39,105 - lava_callback - INFO - Starting async job submit for node 69c397c518a3717aa65d3d51 2026-03-27 08:07:39,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:07:39,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c397c518a3717aa65d3d51 HTTP/1.1" 200 None 2026-03-27 08:07:39,122 - lava_callback - DEBUG - Processing node 69c397c518a3717aa65d3d51: baseline-arm64-clabbe 2026-03-27 08:07:39,122 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c397c518a3717aa65d3d51 2026-03-27 08:07:39,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:07:39,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c397c518a3717aa65d3d51 HTTP/1.1" 200 None 2026-03-27 08:07:39,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:07:39,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-27 08:07:39,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:07:39,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c397c518a3717aa65d3d51 HTTP/1.1" 200 None 2026-03-27 08:07:39,183 - lava_callback - ERROR - Error processing callback for node 69c397c518a3717aa65d3d51: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34788 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60676 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:08:59,100 - lava_callback - INFO - Received callback for node 69c3981218a3717aa65d3d58 INFO: 172.20.0.1:60682 - "POST /node/69c3981218a3717aa65d3d58 HTTP/1.0" 202 Accepted 2026-03-27 08:08:59,106 - lava_callback - INFO - Starting async job submit for node 69c3981218a3717aa65d3d58 2026-03-27 08:08:59,107 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:08:59,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3981218a3717aa65d3d58 HTTP/1.1" 200 None 2026-03-27 08:08:59,135 - lava_callback - DEBUG - Processing node 69c3981218a3717aa65d3d58: baseline-arm64-clabbe 2026-03-27 08:08:59,135 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3981218a3717aa65d3d58 2026-03-27 08:08:59,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:08:59,149 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3981218a3717aa65d3d58 HTTP/1.1" 200 None 2026-03-27 08:08:59,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:08:59,164 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-27 08:08:59,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:08:59,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3981218a3717aa65d3d58 HTTP/1.1" 200 None 2026-03-27 08:08:59,193 - lava_callback - ERROR - Error processing callback for node 69c3981218a3717aa65d3d58: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49932 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38486 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38496 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:10:19,193 - lava_callback - INFO - Received callback for node 69c3985e18a3717aa65d3d68 INFO: 172.20.0.1:38512 - "POST /node/69c3985e18a3717aa65d3d68 HTTP/1.0" 202 Accepted 2026-03-27 08:10:19,199 - lava_callback - INFO - Starting async job submit for node 69c3985e18a3717aa65d3d68 2026-03-27 08:10:19,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:10:19,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3985e18a3717aa65d3d68 HTTP/1.1" 200 None 2026-03-27 08:10:19,233 - lava_callback - DEBUG - Processing node 69c3985e18a3717aa65d3d68: baseline-arm64-clabbe 2026-03-27 08:10:19,234 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3985e18a3717aa65d3d68 2026-03-27 08:10:19,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:10:19,248 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3985e18a3717aa65d3d68 HTTP/1.1" 200 None 2026-03-27 08:10:19,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:10:19,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-27 08:10:19,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:10:19,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3985e18a3717aa65d3d68 HTTP/1.1" 200 None 2026-03-27 08:10:19,322 - lava_callback - ERROR - Error processing callback for node 69c3985e18a3717aa65d3d68: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35954 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56234 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:11:39,215 - lava_callback - INFO - Received callback for node 69c398ab18a3717aa65d3d70 INFO: 172.20.0.1:56250 - "POST /node/69c398ab18a3717aa65d3d70 HTTP/1.0" 202 Accepted 2026-03-27 08:11:39,223 - lava_callback - INFO - Starting async job submit for node 69c398ab18a3717aa65d3d70 2026-03-27 08:11:39,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:11:39,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c398ab18a3717aa65d3d70 HTTP/1.1" 200 None 2026-03-27 08:11:39,264 - lava_callback - DEBUG - Processing node 69c398ab18a3717aa65d3d70: baseline-arm64-clabbe 2026-03-27 08:11:39,264 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c398ab18a3717aa65d3d70 2026-03-27 08:11:39,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:11:39,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c398ab18a3717aa65d3d70 HTTP/1.1" 200 None 2026-03-27 08:11:39,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:11:39,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-27 08:11:39,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:11:39,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c398ab18a3717aa65d3d70 HTTP/1.1" 200 None 2026-03-27 08:11:39,365 - lava_callback - ERROR - Error processing callback for node 69c398ab18a3717aa65d3d70: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42236 - "GET / HTTP/1.0" 200 OK INFO: Shutting down INFO: Waiting for application shutdown. INFO: Application shutdown complete. INFO: Finished server process [1]