INFO: Started server process [1] INFO: Waiting for application startup. INFO: Application startup complete. INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit) INFO: 172.21.0.1:57360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54220 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58682 - "GET / HTTP/1.0" 200 OK 2026-03-21 00:13:35,102 - lava_callback - INFO - Received callback for node 69bb3fa421a77312e22227b0 2026-03-21 00:13:35,109 - lava_callback - INFO - Starting async job submit for node 69bb3fa421a77312e22227b0 INFO: 172.21.0.1:39554 - "POST /node/69bb3fa421a77312e22227b0 HTTP/1.0" 202 Accepted 2026-03-21 00:13:35,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:13:35,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb3fa421a77312e22227b0 HTTP/1.1" 200 None 2026-03-21 00:13:35,136 - lava_callback - DEBUG - Processing node 69bb3fa421a77312e22227b0: baseline-arm64-baylibre 2026-03-21 00:13:35,136 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb3fa421a77312e22227b0 2026-03-21 00:13:35,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:13:35,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb3fa421a77312e22227b0 HTTP/1.1" 200 None 2026-03-21 00:13:35,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:13:35,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7cd7000b80c8ffdcb82ae HTTP/1.1" 200 None 2026-03-21 00:13:35,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:13:35,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb3fa421a77312e22227b0 HTTP/1.1" 200 None 2026-03-21 00:13:35,260 - lava_callback - ERROR - Error processing callback for node 69bb3fa421a77312e22227b0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55774 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58060 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55778 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56868 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47182 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54570 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34316 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51148 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39770 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41256 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34416 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51872 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32788 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:32804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51882 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58584 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46304 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39908 - "GET / HTTP/1.0" 200 OK 2026-03-21 00:30:12,346 - lava_callback - INFO - Received callback for node 69bde617d54b5122738c638e INFO: 172.21.0.1:52142 - "POST /node/69bde617d54b5122738c638e HTTP/1.0" 202 Accepted 2026-03-21 00:30:12,352 - lava_callback - INFO - Starting async job submit for node 69bde617d54b5122738c638e 2026-03-21 00:30:12,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:30:12,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde617d54b5122738c638e HTTP/1.1" 200 None 2026-03-21 00:30:12,407 - lava_callback - DEBUG - Processing node 69bde617d54b5122738c638e: baseline-arm 2026-03-21 00:30:12,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:30:12,816 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:30:12,818 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69bde617d54b5122738c638e/log.txt.gz 2026-03-21 00:30:12,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:30:13,139 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:30:13,141 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69bde617d54b5122738c638e/lava-logs.yaml 2026-03-21 00:30:13,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:30:13,304 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:30:13,306 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69bde617d54b5122738c638e/lava_callback.json.gz 2026-03-21 00:30:13,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:30:13,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde617d54b5122738c638e HTTP/1.1" 200 None 2026-03-21 00:30:13,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:30:13,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde35fd54b5122738c3d99 HTTP/1.1" 200 None 2026-03-21 00:30:13,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:30:13,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bde617d54b5122738c638e HTTP/1.1" 200 None 2026-03-21 00:30:13,405 - lava_callback - INFO - Completed processing callback for node 69bde617d54b5122738c638e INFO: 172.21.0.1:52154 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60362 - "GET / HTTP/1.0" 200 OK 2026-03-21 00:30:35,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:30:35,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:43650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44798 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44976 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53838 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38186 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35040 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45956 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45900 - "GET / HTTP/1.0" 200 OK 2026-03-21 00:36:51,301 - lava_callback - INFO - Received callback for node 69bde809d54b5122738c63b4 INFO: 172.21.0.1:47690 - "POST /node/69bde809d54b5122738c63b4 HTTP/1.0" 202 Accepted 2026-03-21 00:36:51,311 - lava_callback - INFO - Starting async job submit for node 69bde809d54b5122738c63b4 2026-03-21 00:36:51,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:36:51,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde809d54b5122738c63b4 HTTP/1.1" 200 None 2026-03-21 00:36:51,377 - lava_callback - DEBUG - Processing node 69bde809d54b5122738c63b4: baseline-x86 2026-03-21 00:36:51,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:36:51,780 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:36:51,781 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69bde809d54b5122738c63b4/log.txt.gz 2026-03-21 00:36:51,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:36:52,062 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:36:52,064 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69bde809d54b5122738c63b4/lava-logs.yaml 2026-03-21 00:36:52,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:36:52,216 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:36:52,218 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69bde809d54b5122738c63b4/lava_callback.json.gz 2026-03-21 00:36:52,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:36:52,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde809d54b5122738c63b4 HTTP/1.1" 200 None 2026-03-21 00:36:52,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:36:52,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde37fd54b5122738c3dc0 HTTP/1.1" 200 None 2026-03-21 00:36:52,266 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:36:52,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bde809d54b5122738c63b4 HTTP/1.1" 200 None 2026-03-21 00:36:52,300 - lava_callback - INFO - Completed processing callback for node 69bde809d54b5122738c63b4 INFO: 172.21.0.1:47696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33326 - "GET / HTTP/1.0" 200 OK 2026-03-21 00:37:05,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:37:05,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-21 00:37:05,350 - lava_callback - INFO - Received callback for node 69bde80cd54b5122738c63bd INFO: 172.21.0.1:33340 - "POST /node/69bde80cd54b5122738c63bd HTTP/1.0" 202 Accepted 2026-03-21 00:37:05,356 - lava_callback - INFO - Starting async job submit for node 69bde80cd54b5122738c63bd 2026-03-21 00:37:05,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:37:05,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde80cd54b5122738c63bd HTTP/1.1" 200 None 2026-03-21 00:37:05,438 - lava_callback - DEBUG - Processing node 69bde80cd54b5122738c63bd: baseline-x86 2026-03-21 00:37:05,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:37:05,736 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:37:05,737 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69bde80cd54b5122738c63bd/log.txt.gz 2026-03-21 00:37:05,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:37:06,003 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:37:06,005 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69bde80cd54b5122738c63bd/lava-logs.yaml 2026-03-21 00:37:06,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:37:06,148 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:37:06,149 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69bde80cd54b5122738c63bd/lava_callback.json.gz 2026-03-21 00:37:06,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:37:06,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde80cd54b5122738c63bd HTTP/1.1" 200 None 2026-03-21 00:37:06,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:37:06,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde37fd54b5122738c3dc0 HTTP/1.1" 200 None 2026-03-21 00:37:06,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:37:06,248 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bde80cd54b5122738c63bd HTTP/1.1" 200 None 2026-03-21 00:37:06,250 - lava_callback - INFO - Completed processing callback for node 69bde80cd54b5122738c63bd INFO: 172.21.0.1:33354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33246 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55904 - "GET / HTTP/1.0" 200 OK 2026-03-21 00:37:35,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:37:35,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:48902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48658 - "GET / HTTP/1.0" 200 OK 2026-03-21 00:37:46,570 - lava_callback - INFO - Received callback for node 69bde80ad54b5122738c63b7 INFO: 172.21.0.1:48670 - "POST /node/69bde80ad54b5122738c63b7 HTTP/1.0" 202 Accepted 2026-03-21 00:37:46,579 - lava_callback - INFO - Starting async job submit for node 69bde80ad54b5122738c63b7 2026-03-21 00:37:46,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:37:46,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde80ad54b5122738c63b7 HTTP/1.1" 200 None 2026-03-21 00:37:46,651 - lava_callback - DEBUG - Processing node 69bde80ad54b5122738c63b7: baseline-x86 2026-03-21 00:37:47,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:37:47,414 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:37:47,416 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69bde80ad54b5122738c63b7/log.txt.gz 2026-03-21 00:37:47,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:37:48,087 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:37:48,089 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69bde80ad54b5122738c63b7/lava-logs.yaml 2026-03-21 00:37:48,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:37:48,246 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:37:48,248 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69bde80ad54b5122738c63b7/lava_callback.json.gz 2026-03-21 00:37:48,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:37:48,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde80ad54b5122738c63b7 HTTP/1.1" 200 None 2026-03-21 00:37:48,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:37:48,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde37fd54b5122738c3dc0 HTTP/1.1" 200 None 2026-03-21 00:37:48,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:37:48,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bde80ad54b5122738c63b7 HTTP/1.1" 200 None 2026-03-21 00:37:48,337 - lava_callback - INFO - Completed processing callback for node 69bde80ad54b5122738c63b7 INFO: 172.21.0.1:37610 - "GET / HTTP/1.0" 200 OK 2026-03-21 00:38:03,143 - lava_callback - INFO - Received callback for node 69bde80dd54b5122738c63c0 INFO: 172.21.0.1:44520 - "POST /node/69bde80dd54b5122738c63c0 HTTP/1.0" 202 Accepted 2026-03-21 00:38:03,150 - lava_callback - INFO - Starting async job submit for node 69bde80dd54b5122738c63c0 2026-03-21 00:38:03,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:38:03,193 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde80dd54b5122738c63c0 HTTP/1.1" 200 None 2026-03-21 00:38:03,194 - lava_callback - DEBUG - Processing node 69bde80dd54b5122738c63c0: baseline-x86 2026-03-21 00:38:03,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:44532 - "GET / HTTP/1.0" 200 OK 2026-03-21 00:38:03,808 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:38:03,810 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69bde80dd54b5122738c63c0/log.txt.gz 2026-03-21 00:38:04,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:38:04,407 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:38:04,409 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69bde80dd54b5122738c63c0/lava-logs.yaml 2026-03-21 00:38:04,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:38:04,559 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:38:04,561 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69bde80dd54b5122738c63c0/lava_callback.json.gz 2026-03-21 00:38:04,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:38:04,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde80dd54b5122738c63c0 HTTP/1.1" 200 None 2026-03-21 00:38:04,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:38:04,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde37fd54b5122738c3dc0 HTTP/1.1" 200 None 2026-03-21 00:38:04,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:38:04,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bde80dd54b5122738c63c0 HTTP/1.1" 200 None 2026-03-21 00:38:04,666 - lava_callback - INFO - Completed processing callback for node 69bde80dd54b5122738c63c0 2026-03-21 00:38:05,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:38:05,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:44542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58562 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52844 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52860 - "GET / HTTP/1.0" 200 OK 2026-03-21 00:39:19,256 - lava_callback - INFO - Received callback for node 69bde8a6d54b5122738c63f7 INFO: 172.21.0.1:52872 - "POST /node/69bde8a6d54b5122738c63f7 HTTP/1.0" 202 Accepted 2026-03-21 00:39:19,263 - lava_callback - INFO - Starting async job submit for node 69bde8a6d54b5122738c63f7 2026-03-21 00:39:19,266 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:39:19,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde8a6d54b5122738c63f7 HTTP/1.1" 200 None 2026-03-21 00:39:19,285 - lava_callback - DEBUG - Processing node 69bde8a6d54b5122738c63f7: baseline-arm 2026-03-21 00:39:19,293 - lava_callback - WARNING - Failed to upload log 2026-03-21 00:39:19,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:39:19,517 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:39:19,518 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69bde8a6d54b5122738c63f7/lava-logs.yaml 2026-03-21 00:39:19,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:39:19,649 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:39:19,651 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69bde8a6d54b5122738c63f7/lava_callback.json.gz 2026-03-21 00:39:19,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:39:19,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde8a6d54b5122738c63f7 HTTP/1.1" 200 None 2026-03-21 00:39:19,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:39:19,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-21 00:39:19,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:39:19,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bde8a6d54b5122738c63f7 HTTP/1.1" 200 None 2026-03-21 00:39:19,722 - lava_callback - INFO - Completed processing callback for node 69bde8a6d54b5122738c63f7 INFO: 172.21.0.1:49406 - "GET / HTTP/1.0" 200 OK 2026-03-21 00:39:33,682 - lava_callback - INFO - Received callback for node 69bde8a7d54b5122738c63fa INFO: 172.21.0.1:39716 - "POST /node/69bde8a7d54b5122738c63fa HTTP/1.0" 202 Accepted 2026-03-21 00:39:33,730 - lava_callback - INFO - Starting async job submit for node 69bde8a7d54b5122738c63fa 2026-03-21 00:39:33,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:39:33,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde8a7d54b5122738c63fa HTTP/1.1" 200 None 2026-03-21 00:39:33,765 - lava_callback - DEBUG - Processing node 69bde8a7d54b5122738c63fa: baseline-arm 2026-03-21 00:39:33,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:39:34,070 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:39:34,071 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69bde8a7d54b5122738c63fa/log.txt.gz 2026-03-21 00:39:34,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:39:34,328 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:39:34,330 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69bde8a7d54b5122738c63fa/lava-logs.yaml 2026-03-21 00:39:34,334 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:39:34,454 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:39:34,456 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69bde8a7d54b5122738c63fa/lava_callback.json.gz 2026-03-21 00:39:34,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:39:34,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde8a7d54b5122738c63fa HTTP/1.1" 200 None 2026-03-21 00:39:34,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:39:34,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-21 00:39:34,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:39:34,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bde8a7d54b5122738c63fa HTTP/1.1" 200 None 2026-03-21 00:39:34,536 - lava_callback - INFO - Completed processing callback for node 69bde8a7d54b5122738c63fa 2026-03-21 00:39:35,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:39:35,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:39720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42622 - "GET / HTTP/1.0" 200 OK 2026-03-21 00:39:54,502 - lava_callback - INFO - Received callback for node 69bde8aad54b5122738c6403 INFO: 172.21.0.1:45504 - "POST /node/69bde8aad54b5122738c6403 HTTP/1.0" 202 Accepted 2026-03-21 00:39:54,562 - lava_callback - INFO - Starting async job submit for node 69bde8aad54b5122738c6403 2026-03-21 00:39:54,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:39:54,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde8aad54b5122738c6403 HTTP/1.1" 200 None 2026-03-21 00:39:54,622 - lava_callback - DEBUG - Processing node 69bde8aad54b5122738c6403: baseline-arm 2026-03-21 00:39:54,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:39:55,370 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:39:55,371 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69bde8aad54b5122738c6403/log.txt.gz 2026-03-21 00:39:55,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:39:55,876 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:39:55,878 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69bde8aad54b5122738c6403/lava-logs.yaml 2026-03-21 00:39:55,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:39:56,039 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:39:56,041 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69bde8aad54b5122738c6403/lava_callback.json.gz 2026-03-21 00:39:56,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:39:56,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde8aad54b5122738c6403 HTTP/1.1" 200 None 2026-03-21 00:39:56,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:39:56,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-21 00:39:56,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:39:56,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bde8aad54b5122738c6403 HTTP/1.1" 200 None 2026-03-21 00:39:56,191 - lava_callback - INFO - Completed processing callback for node 69bde8aad54b5122738c6403 INFO: 172.21.0.1:45506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44070 - "GET / HTTP/1.0" 200 OK 2026-03-21 00:40:05,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:40:05,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:44072 - "GET / HTTP/1.0" 200 OK 2026-03-21 00:40:11,831 - lava_callback - INFO - Received callback for node 69bde8a9d54b5122738c6400 INFO: 172.21.0.1:59412 - "POST /node/69bde8a9d54b5122738c6400 HTTP/1.0" 202 Accepted 2026-03-21 00:40:11,841 - lava_callback - INFO - Starting async job submit for node 69bde8a9d54b5122738c6400 2026-03-21 00:40:11,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:40:11,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde8a9d54b5122738c6400 HTTP/1.1" 200 None 2026-03-21 00:40:11,902 - lava_callback - DEBUG - Processing node 69bde8a9d54b5122738c6400: baseline-arm 2026-03-21 00:40:12,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:40:12,309 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:40:12,311 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69bde8a9d54b5122738c6400/log.txt.gz 2026-03-21 00:40:12,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:40:12,616 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:40:12,618 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69bde8a9d54b5122738c6400/lava-logs.yaml 2026-03-21 00:40:12,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:40:12,818 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:40:12,820 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69bde8a9d54b5122738c6400/lava_callback.json.gz 2026-03-21 00:40:12,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:40:12,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde8a9d54b5122738c6400 HTTP/1.1" 200 None 2026-03-21 00:40:12,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:40:12,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-21 00:40:12,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:40:12,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bde8a9d54b5122738c6400 HTTP/1.1" 200 None 2026-03-21 00:40:12,908 - lava_callback - INFO - Completed processing callback for node 69bde8a9d54b5122738c6400 INFO: 172.21.0.1:59426 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55514 - "GET / HTTP/1.0" 200 OK 2026-03-21 00:40:35,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:40:35,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-21 00:40:35,746 - lava_callback - INFO - Received callback for node 69bde8a6d54b5122738c63f4 INFO: 172.21.0.1:40322 - "POST /node/69bde8a6d54b5122738c63f4 HTTP/1.0" 202 Accepted 2026-03-21 00:40:35,752 - lava_callback - INFO - Starting async job submit for node 69bde8a6d54b5122738c63f4 2026-03-21 00:40:35,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:40:35,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde8a6d54b5122738c63f4 HTTP/1.1" 200 None 2026-03-21 00:40:35,793 - lava_callback - DEBUG - Processing node 69bde8a6d54b5122738c63f4: baseline-arm 2026-03-21 00:40:35,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:40:36,126 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:40:36,128 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69bde8a6d54b5122738c63f4/log.txt.gz 2026-03-21 00:40:36,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:40:36,446 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:40:36,448 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69bde8a6d54b5122738c63f4/lava-logs.yaml 2026-03-21 00:40:36,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:40326 - "GET / HTTP/1.0" 200 OK 2026-03-21 00:40:36,590 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:40:36,592 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69bde8a6d54b5122738c63f4/lava_callback.json.gz 2026-03-21 00:40:36,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:40:36,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde8a6d54b5122738c63f4 HTTP/1.1" 200 None 2026-03-21 00:40:36,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:40:36,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-21 00:40:36,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:40:36,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bde8a6d54b5122738c63f4 HTTP/1.1" 200 None 2026-03-21 00:40:36,690 - lava_callback - INFO - Completed processing callback for node 69bde8a6d54b5122738c63f4 INFO: 172.21.0.1:51300 - "GET / HTTP/1.0" 200 OK 2026-03-21 00:40:47,936 - lava_callback - INFO - Received callback for node 69bde8c7d54b5122738c6479 INFO: 172.21.0.1:51304 - "POST /node/69bde8c7d54b5122738c6479 HTTP/1.0" 202 Accepted 2026-03-21 00:40:47,985 - lava_callback - INFO - Starting async job submit for node 69bde8c7d54b5122738c6479 2026-03-21 00:40:48,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:40:48,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde8c7d54b5122738c6479 HTTP/1.1" 200 None 2026-03-21 00:40:48,021 - lava_callback - DEBUG - Processing node 69bde8c7d54b5122738c6479: baseline-arm-pengutronix 2026-03-21 00:40:48,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:40:48,266 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:40:48,268 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69bde8c7d54b5122738c6479/log.txt.gz 2026-03-21 00:40:48,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:40:48,506 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:40:48,507 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69bde8c7d54b5122738c6479/lava-logs.yaml 2026-03-21 00:40:48,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:40:48,624 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:40:48,626 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69bde8c7d54b5122738c6479/lava_callback.json.gz 2026-03-21 00:40:48,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:40:48,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde8c7d54b5122738c6479 HTTP/1.1" 200 None 2026-03-21 00:40:48,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:40:48,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-21 00:40:48,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:40:48,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bde8c7d54b5122738c6479 HTTP/1.1" 200 None 2026-03-21 00:40:48,704 - lava_callback - INFO - Completed processing callback for node 69bde8c7d54b5122738c6479 2026-03-21 00:40:50,628 - lava_callback - INFO - Received callback for node 69bde8b2d54b5122738c6412 INFO: 172.21.0.1:50750 - "POST /node/69bde8b2d54b5122738c6412 HTTP/1.0" 202 Accepted 2026-03-21 00:40:50,634 - lava_callback - INFO - Starting async job submit for node 69bde8b2d54b5122738c6412 2026-03-21 00:40:50,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:40:50,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde8b2d54b5122738c6412 HTTP/1.1" 200 None 2026-03-21 00:40:50,670 - lava_callback - DEBUG - Processing node 69bde8b2d54b5122738c6412: baseline-arm-broonie 2026-03-21 00:40:50,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:40:50,980 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:40:50,981 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69bde8b2d54b5122738c6412/log.txt.gz 2026-03-21 00:40:51,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:40:51,277 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:40:51,279 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69bde8b2d54b5122738c6412/lava-logs.yaml 2026-03-21 00:40:51,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:40:51,408 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:40:51,410 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69bde8b2d54b5122738c6412/lava_callback.json.gz 2026-03-21 00:40:51,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:40:51,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde8b2d54b5122738c6412 HTTP/1.1" 200 None 2026-03-21 00:40:51,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:40:51,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-21 00:40:51,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:40:51,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bde8b2d54b5122738c6412 HTTP/1.1" 200 None 2026-03-21 00:40:51,498 - lava_callback - INFO - Completed processing callback for node 69bde8b2d54b5122738c6412 2026-03-21 00:40:52,329 - lava_callback - INFO - Received callback for node 69bde8b4d54b5122738c6419 INFO: 172.21.0.1:50764 - "POST /node/69bde8b4d54b5122738c6419 HTTP/1.0" 202 Accepted 2026-03-21 00:40:52,335 - lava_callback - INFO - Starting async job submit for node 69bde8b4d54b5122738c6419 2026-03-21 00:40:52,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:40:52,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde8b4d54b5122738c6419 HTTP/1.1" 200 None 2026-03-21 00:40:52,376 - lava_callback - DEBUG - Processing node 69bde8b4d54b5122738c6419: baseline-arm-broonie 2026-03-21 00:40:52,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:40:52,729 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:40:52,731 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69bde8b4d54b5122738c6419/log.txt.gz 2026-03-21 00:40:52,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:40:53,030 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:40:53,032 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69bde8b4d54b5122738c6419/lava-logs.yaml 2026-03-21 00:40:53,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:40:53,166 - lava_callback - INFO - Received callback for node 69bde8c4d54b5122738c646f 2026-03-21 00:40:53,173 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:40:53,173 - lava_callback - INFO - Starting async job submit for node 69bde8c4d54b5122738c646f INFO: 172.21.0.1:50778 - "POST /node/69bde8c4d54b5122738c646f HTTP/1.0" 202 Accepted 2026-03-21 00:40:53,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:40:53,194 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69bde8b4d54b5122738c6419/lava_callback.json.gz 2026-03-21 00:40:53,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:40:53,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde8b4d54b5122738c6419 HTTP/1.1" 200 None 2026-03-21 00:40:53,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:40:53,236 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde8c4d54b5122738c646f HTTP/1.1" 200 None 2026-03-21 00:40:53,237 - lava_callback - DEBUG - Processing node 69bde8c4d54b5122738c646f: baseline-arm-pengutronix 2026-03-21 00:40:53,302 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-21 00:40:53,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:40:53,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:40:53,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bde8b4d54b5122738c6419 HTTP/1.1" 200 None 2026-03-21 00:40:53,470 - lava_callback - INFO - Completed processing callback for node 69bde8b4d54b5122738c6419 2026-03-21 00:40:53,616 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:40:53,618 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69bde8c4d54b5122738c646f/log.txt.gz 2026-03-21 00:40:53,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:40:53,930 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:40:53,932 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69bde8c4d54b5122738c646f/lava-logs.yaml 2026-03-21 00:40:53,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:40:54,059 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:40:54,061 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69bde8c4d54b5122738c646f/lava_callback.json.gz 2026-03-21 00:40:54,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:40:54,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde8c4d54b5122738c646f HTTP/1.1" 200 None 2026-03-21 00:40:54,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:40:54,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-21 00:40:54,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:40:54,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bde8c4d54b5122738c646f HTTP/1.1" 200 None 2026-03-21 00:40:54,159 - lava_callback - INFO - Completed processing callback for node 69bde8c4d54b5122738c646f INFO: 172.21.0.1:50784 - "GET / HTTP/1.0" 200 OK 2026-03-21 00:40:58,329 - lava_callback - INFO - Received callback for node 69bde8b5d54b5122738c641c INFO: 172.21.0.1:50798 - "POST /node/69bde8b5d54b5122738c641c HTTP/1.0" 202 Accepted 2026-03-21 00:40:58,335 - lava_callback - INFO - Starting async job submit for node 69bde8b5d54b5122738c641c 2026-03-21 00:40:58,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:40:58,374 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde8b5d54b5122738c641c HTTP/1.1" 200 None 2026-03-21 00:40:58,375 - lava_callback - DEBUG - Processing node 69bde8b5d54b5122738c641c: baseline-arm-broonie 2026-03-21 00:40:58,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:40:58,710 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:40:58,711 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69bde8b5d54b5122738c641c/log.txt.gz 2026-03-21 00:40:58,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:40:59,012 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:40:59,014 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69bde8b5d54b5122738c641c/lava-logs.yaml 2026-03-21 00:40:59,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:40:59,150 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:40:59,152 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69bde8b5d54b5122738c641c/lava_callback.json.gz 2026-03-21 00:40:59,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:40:59,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde8b5d54b5122738c641c HTTP/1.1" 200 None 2026-03-21 00:40:59,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:40:59,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-21 00:40:59,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:40:59,254 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bde8b5d54b5122738c641c HTTP/1.1" 200 None 2026-03-21 00:40:59,256 - lava_callback - INFO - Completed processing callback for node 69bde8b5d54b5122738c641c 2026-03-21 00:40:59,731 - lava_callback - INFO - Received callback for node 69bde8b9d54b5122738c644b INFO: 172.21.0.1:46888 - "POST /node/69bde8b9d54b5122738c644b HTTP/1.0" 202 Accepted 2026-03-21 00:40:59,737 - lava_callback - INFO - Starting async job submit for node 69bde8b9d54b5122738c644b 2026-03-21 00:40:59,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:40:59,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde8b9d54b5122738c644b HTTP/1.1" 200 None 2026-03-21 00:40:59,776 - lava_callback - DEBUG - Processing node 69bde8b9d54b5122738c644b: baseline-arm-clabbe 2026-03-21 00:40:59,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:41:00,107 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:41:00,109 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69bde8b9d54b5122738c644b/log.txt.gz 2026-03-21 00:41:00,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:41:00,409 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:41:00,411 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69bde8b9d54b5122738c644b/lava-logs.yaml 2026-03-21 00:41:00,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:41:00,622 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:41:00,623 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69bde8b9d54b5122738c644b/lava_callback.json.gz 2026-03-21 00:41:00,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:41:00,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde8b9d54b5122738c644b HTTP/1.1" 200 None 2026-03-21 00:41:00,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:41:00,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-21 00:41:00,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:41:00,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bde8b9d54b5122738c644b HTTP/1.1" 200 None 2026-03-21 00:41:00,712 - lava_callback - INFO - Completed processing callback for node 69bde8b9d54b5122738c644b INFO: 172.21.0.1:46898 - "GET / HTTP/1.0" 200 OK 2026-03-21 00:41:05,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:41:05,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.21.0.1:46906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47846 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-21 00:41:15,686 - lava_callback - INFO - Received callback for node 69bde922d54b5122738c6541 INFO: 172.21.0.1:47860 - "POST /node/69bde922d54b5122738c6541 HTTP/1.0" 202 Accepted 2026-03-21 00:41:15,692 - lava_callback - INFO - Starting async job submit for node 69bde922d54b5122738c6541 2026-03-21 00:41:15,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:41:15,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde922d54b5122738c6541 HTTP/1.1" 200 None 2026-03-21 00:41:15,710 - lava_callback - DEBUG - Processing node 69bde922d54b5122738c6541: baseline-arm 2026-03-21 00:41:15,715 - lava_callback - WARNING - Failed to upload log 2026-03-21 00:41:15,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:41:15,832 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:41:15,835 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69bde922d54b5122738c6541/lava-logs.yaml 2026-03-21 00:41:15,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:41:15,984 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:41:15,985 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69bde922d54b5122738c6541/lava_callback.json.gz 2026-03-21 00:41:15,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:41:16,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde922d54b5122738c6541 HTTP/1.1" 200 None 2026-03-21 00:41:16,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:41:16,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-21 00:41:16,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:41:16,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bde922d54b5122738c6541 HTTP/1.1" 200 None 2026-03-21 00:41:16,046 - lava_callback - INFO - Completed processing callback for node 69bde922d54b5122738c6541 INFO: 172.21.0.1:47872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39986 - "GET / HTTP/1.0" 200 OK 2026-03-21 00:41:35,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:41:35,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:38914 - "GET / HTTP/1.0" 200 OK 2026-03-21 00:41:43,071 - lava_callback - INFO - Received callback for node 69bde8ded54b5122738c64b3 INFO: 172.21.0.1:55124 - "POST /node/69bde8ded54b5122738c64b3 HTTP/1.0" 202 Accepted 2026-03-21 00:41:43,077 - lava_callback - INFO - Starting async job submit for node 69bde8ded54b5122738c64b3 2026-03-21 00:41:43,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:41:43,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde8ded54b5122738c64b3 HTTP/1.1" 200 None 2026-03-21 00:41:43,115 - lava_callback - DEBUG - Processing node 69bde8ded54b5122738c64b3: ltp-dio 2026-03-21 00:41:43,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:41:43,659 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:41:43,661 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-dio-69bde8ded54b5122738c64b3/log.txt.gz 2026-03-21 00:41:43,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:41:44,099 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:41:44,101 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-dio-69bde8ded54b5122738c64b3/lava-logs.yaml 2026-03-21 00:41:44,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:41:44,283 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:41:44,285 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-dio-69bde8ded54b5122738c64b3/lava_callback.json.gz 2026-03-21 00:41:44,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:41:44,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde8ded54b5122738c64b3 HTTP/1.1" 200 None 2026-03-21 00:41:44,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:41:44,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-21 00:41:44,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:41:44,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bde8ded54b5122738c64b3 HTTP/1.1" 200 None 2026-03-21 00:41:44,374 - lava_callback - INFO - Completed processing callback for node 69bde8ded54b5122738c64b3 INFO: 172.21.0.1:55126 - "GET / HTTP/1.0" 200 OK 2026-03-21 00:41:55,192 - lava_callback - INFO - Received callback for node 69bde8c3d54b5122738c646c INFO: 172.21.0.1:40036 - "POST /node/69bde8c3d54b5122738c646c HTTP/1.0" 202 Accepted 2026-03-21 00:41:55,198 - lava_callback - INFO - Starting async job submit for node 69bde8c3d54b5122738c646c 2026-03-21 00:41:55,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:41:55,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde8c3d54b5122738c646c HTTP/1.1" 200 None 2026-03-21 00:41:55,245 - lava_callback - DEBUG - Processing node 69bde8c3d54b5122738c646c: baseline-arm-pengutronix 2026-03-21 00:41:55,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:41:55,491 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:41:55,493 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69bde8c3d54b5122738c646c/log.txt.gz 2026-03-21 00:41:55,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:41:55,765 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:41:55,767 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69bde8c3d54b5122738c646c/lava-logs.yaml 2026-03-21 00:41:55,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:41:55,897 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:41:55,899 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69bde8c3d54b5122738c646c/lava_callback.json.gz 2026-03-21 00:41:55,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:41:55,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde8c3d54b5122738c646c HTTP/1.1" 200 None 2026-03-21 00:41:55,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:41:55,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-21 00:41:55,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:41:56,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bde8c3d54b5122738c646c HTTP/1.1" 200 None 2026-03-21 00:41:56,065 - lava_callback - INFO - Completed processing callback for node 69bde8c3d54b5122738c646c INFO: 172.21.0.1:40048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43474 - "GET / HTTP/1.0" 200 OK 2026-03-21 00:42:05,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:42:05,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:43480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57216 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60388 - "GET / HTTP/1.0" 200 OK 2026-03-21 00:42:50,798 - lava_callback - INFO - Received callback for node 69bde8bad54b5122738c644f INFO: 172.21.0.1:58812 - "POST /node/69bde8bad54b5122738c644f HTTP/1.0" 202 Accepted 2026-03-21 00:42:50,804 - lava_callback - INFO - Starting async job submit for node 69bde8bad54b5122738c644f 2026-03-21 00:42:50,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:42:50,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde8bad54b5122738c644f HTTP/1.1" 200 None 2026-03-21 00:42:50,841 - lava_callback - DEBUG - Processing node 69bde8bad54b5122738c644f: baseline-arm-clabbe 2026-03-21 00:42:51,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:42:51,236 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:42:51,237 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69bde8bad54b5122738c644f/log.txt.gz 2026-03-21 00:42:51,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:42:51,558 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:42:51,560 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69bde8bad54b5122738c644f/lava-logs.yaml 2026-03-21 00:42:51,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:42:51,715 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:42:51,717 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69bde8bad54b5122738c644f/lava_callback.json.gz 2026-03-21 00:42:51,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:42:51,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde8bad54b5122738c644f HTTP/1.1" 200 None 2026-03-21 00:42:51,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:42:51,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-21 00:42:51,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:42:51,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bde8bad54b5122738c644f HTTP/1.1" 200 None 2026-03-21 00:42:51,807 - lava_callback - INFO - Completed processing callback for node 69bde8bad54b5122738c644f INFO: 172.21.0.1:58826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42736 - "GET / HTTP/1.0" 200 OK 2026-03-21 00:43:05,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:43:05,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:42740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56882 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42308 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42322 - "GET / HTTP/1.0" 200 OK 2026-03-21 00:44:20,029 - lava_callback - INFO - Received callback for node 69bde8ebd54b5122738c64d0 INFO: 172.21.0.1:36762 - "POST /node/69bde8ebd54b5122738c64d0 HTTP/1.0" 202 Accepted 2026-03-21 00:44:20,036 - lava_callback - INFO - Starting async job submit for node 69bde8ebd54b5122738c64d0 2026-03-21 00:44:20,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:44:20,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde8ebd54b5122738c64d0 HTTP/1.1" 200 None 2026-03-21 00:44:20,076 - lava_callback - DEBUG - Processing node 69bde8ebd54b5122738c64d0: ltp-syscalls 2026-03-21 00:44:20,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:44:21,004 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:44:21,005 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-69bde8ebd54b5122738c64d0/log.txt.gz 2026-03-21 00:44:21,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:44:21,806 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:44:21,809 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-69bde8ebd54b5122738c64d0/lava-logs.yaml 2026-03-21 00:44:21,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:44:21,979 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:44:21,980 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-69bde8ebd54b5122738c64d0/lava_callback.json.gz 2026-03-21 00:44:22,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:44:22,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde8ebd54b5122738c64d0 HTTP/1.1" 200 None 2026-03-21 00:44:22,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:44:22,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-21 00:44:22,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:44:22,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bde8ebd54b5122738c64d0 HTTP/1.1" 200 None 2026-03-21 00:44:22,062 - lava_callback - INFO - Completed processing callback for node 69bde8ebd54b5122738c64d0 INFO: 172.21.0.1:36776 - "GET / HTTP/1.0" 200 OK 2026-03-21 00:44:35,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:44:35,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:51056 - "GET / HTTP/1.0" 200 OK 2026-03-21 00:44:45,884 - lava_callback - INFO - Received callback for node 69bde8bdd54b5122738c645b INFO: 172.21.0.1:57460 - "POST /node/69bde8bdd54b5122738c645b HTTP/1.0" 202 Accepted 2026-03-21 00:44:45,891 - lava_callback - INFO - Starting async job submit for node 69bde8bdd54b5122738c645b 2026-03-21 00:44:45,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:44:45,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde8bdd54b5122738c645b HTTP/1.1" 200 None 2026-03-21 00:44:45,950 - lava_callback - DEBUG - Processing node 69bde8bdd54b5122738c645b: baseline-arm-clabbe 2026-03-21 00:44:46,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:44:46,476 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:44:46,478 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69bde8bdd54b5122738c645b/log.txt.gz INFO: 172.21.0.1:57466 - "GET / HTTP/1.0" 200 OK 2026-03-21 00:44:46,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:44:46,910 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:44:46,913 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69bde8bdd54b5122738c645b/lava-logs.yaml 2026-03-21 00:44:46,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:44:47,132 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:44:47,134 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69bde8bdd54b5122738c645b/lava_callback.json.gz 2026-03-21 00:44:47,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:44:47,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde8bdd54b5122738c645b HTTP/1.1" 200 None 2026-03-21 00:44:47,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:44:47,193 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-21 00:44:47,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:44:47,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bde8bdd54b5122738c645b HTTP/1.1" 200 None 2026-03-21 00:44:47,234 - lava_callback - INFO - Completed processing callback for node 69bde8bdd54b5122738c645b INFO: 172.21.0.1:54112 - "GET / HTTP/1.0" 200 OK 2026-03-21 00:44:58,162 - lava_callback - INFO - Received callback for node 69bde80bd54b5122738c63ba INFO: 172.21.0.1:54122 - "POST /node/69bde80bd54b5122738c63ba HTTP/1.0" 202 Accepted 2026-03-21 00:44:58,185 - lava_callback - INFO - Starting async job submit for node 69bde80bd54b5122738c63ba 2026-03-21 00:44:58,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:44:58,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde80bd54b5122738c63ba HTTP/1.1" 200 None 2026-03-21 00:44:58,281 - lava_callback - DEBUG - Processing node 69bde80bd54b5122738c63ba: baseline-x86 2026-03-21 00:44:58,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:44:59,177 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:44:59,179 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69bde80bd54b5122738c63ba/log.txt.gz 2026-03-21 00:44:59,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:44:59,952 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:44:59,954 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69bde80bd54b5122738c63ba/lava-logs.yaml 2026-03-21 00:44:59,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:45:00,193 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:45:00,195 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69bde80bd54b5122738c63ba/lava_callback.json.gz 2026-03-21 00:45:00,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:45:00,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde80bd54b5122738c63ba HTTP/1.1" 200 None 2026-03-21 00:45:00,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:45:00,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde37fd54b5122738c3dc0 HTTP/1.1" 200 None 2026-03-21 00:45:00,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:45:00,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bde80bd54b5122738c63ba HTTP/1.1" 200 None 2026-03-21 00:45:00,342 - lava_callback - INFO - Completed processing callback for node 69bde80bd54b5122738c63ba INFO: 172.21.0.1:37896 - "GET / HTTP/1.0" 200 OK 2026-03-21 00:45:05,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:45:05,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:37910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46108 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37024 - "GET / HTTP/1.0" 200 OK 2026-03-21 00:45:44,605 - lava_callback - INFO - Received callback for node 69bdea2cd54b5122738c65e1 INFO: 172.21.0.1:58892 - "POST /node/69bdea2cd54b5122738c65e1 HTTP/1.0" 202 Accepted 2026-03-21 00:45:44,615 - lava_callback - INFO - Starting async job submit for node 69bdea2cd54b5122738c65e1 2026-03-21 00:45:44,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:45:44,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdea2cd54b5122738c65e1 HTTP/1.1" 200 None 2026-03-21 00:45:44,648 - lava_callback - DEBUG - Processing node 69bdea2cd54b5122738c65e1: baseline-arm 2026-03-21 00:45:44,657 - lava_callback - WARNING - Failed to upload log 2026-03-21 00:45:44,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:45:44,840 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:45:44,842 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69bdea2cd54b5122738c65e1/lava-logs.yaml 2026-03-21 00:45:44,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:45:44,985 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:45:44,987 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69bdea2cd54b5122738c65e1/lava_callback.json.gz 2026-03-21 00:45:44,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:45:45,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdea2cd54b5122738c65e1 HTTP/1.1" 200 None 2026-03-21 00:45:45,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:45:45,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-21 00:45:45,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:45:45,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdea2cd54b5122738c65e1 HTTP/1.1" 200 None 2026-03-21 00:45:45,051 - lava_callback - INFO - Completed processing callback for node 69bdea2cd54b5122738c65e1 INFO: 172.21.0.1:58908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42774 - "GET / HTTP/1.0" 200 OK 2026-03-21 00:46:05,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:46:05,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:42778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45440 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45456 - "GET / HTTP/1.0" 200 OK 2026-03-21 00:46:17,019 - lava_callback - INFO - Received callback for node 69bb474fa1e2bfb1b3346a57 INFO: 172.21.0.1:45468 - "POST /node/69bb474fa1e2bfb1b3346a57 HTTP/1.0" 202 Accepted 2026-03-21 00:46:17,025 - lava_callback - INFO - Starting async job submit for node 69bb474fa1e2bfb1b3346a57 2026-03-21 00:46:17,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:46:17,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb474fa1e2bfb1b3346a57 HTTP/1.1" 200 None 2026-03-21 00:46:17,042 - lava_callback - DEBUG - Processing node 69bb474fa1e2bfb1b3346a57: baseline-arm-baylibre 2026-03-21 00:46:17,043 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb474fa1e2bfb1b3346a57 2026-03-21 00:46:17,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:46:17,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb474fa1e2bfb1b3346a57 HTTP/1.1" 200 None 2026-03-21 00:46:17,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:46:17,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-21 00:46:17,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:46:17,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb474fa1e2bfb1b3346a57 HTTP/1.1" 200 None 2026-03-21 00:46:17,176 - lava_callback - ERROR - Error processing callback for node 69bb474fa1e2bfb1b3346a57: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-21 00:46:17,661 - lava_callback - INFO - Received callback for node 69bb4750a1e2bfb1b3346a5a INFO: 172.21.0.1:45470 - "POST /node/69bb4750a1e2bfb1b3346a5a HTTP/1.0" 202 Accepted 2026-03-21 00:46:17,668 - lava_callback - INFO - Starting async job submit for node 69bb4750a1e2bfb1b3346a5a 2026-03-21 00:46:17,669 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:46:17,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4750a1e2bfb1b3346a5a HTTP/1.1" 200 None 2026-03-21 00:46:17,686 - lava_callback - DEBUG - Processing node 69bb4750a1e2bfb1b3346a5a: baseline-arm-baylibre 2026-03-21 00:46:17,686 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb4750a1e2bfb1b3346a5a 2026-03-21 00:46:17,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:46:17,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4750a1e2bfb1b3346a5a HTTP/1.1" 200 None 2026-03-21 00:46:17,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:46:17,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-21 00:46:17,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:46:17,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb4750a1e2bfb1b3346a5a HTTP/1.1" 200 None 2026-03-21 00:46:17,756 - lava_callback - ERROR - Error processing callback for node 69bb4750a1e2bfb1b3346a5a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-21 00:46:18,306 - lava_callback - INFO - Received callback for node 69bb4752a1e2bfb1b3346a5d INFO: 172.21.0.1:45476 - "POST /node/69bb4752a1e2bfb1b3346a5d HTTP/1.0" 202 Accepted 2026-03-21 00:46:18,314 - lava_callback - INFO - Starting async job submit for node 69bb4752a1e2bfb1b3346a5d 2026-03-21 00:46:18,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:46:18,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4752a1e2bfb1b3346a5d HTTP/1.1" 200 None 2026-03-21 00:46:18,334 - lava_callback - DEBUG - Processing node 69bb4752a1e2bfb1b3346a5d: baseline-arm-baylibre 2026-03-21 00:46:18,334 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb4752a1e2bfb1b3346a5d 2026-03-21 00:46:18,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:46:18,356 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4752a1e2bfb1b3346a5d HTTP/1.1" 200 None 2026-03-21 00:46:18,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:46:18,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-21 00:46:18,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:46:18,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb4752a1e2bfb1b3346a5d HTTP/1.1" 200 None 2026-03-21 00:46:18,399 - lava_callback - ERROR - Error processing callback for node 69bb4752a1e2bfb1b3346a5d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-21 00:46:18,986 - lava_callback - INFO - Received callback for node 69bb4753a1e2bfb1b3346a60 INFO: 172.21.0.1:45482 - "POST /node/69bb4753a1e2bfb1b3346a60 HTTP/1.0" 202 Accepted 2026-03-21 00:46:18,994 - lava_callback - INFO - Starting async job submit for node 69bb4753a1e2bfb1b3346a60 2026-03-21 00:46:19,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:46:19,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4753a1e2bfb1b3346a60 HTTP/1.1" 200 None 2026-03-21 00:46:19,032 - lava_callback - DEBUG - Processing node 69bb4753a1e2bfb1b3346a60: baseline-arm-baylibre 2026-03-21 00:46:19,032 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb4753a1e2bfb1b3346a60 2026-03-21 00:46:19,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:46:19,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4753a1e2bfb1b3346a60 HTTP/1.1" 200 None 2026-03-21 00:46:19,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:46:19,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-21 00:46:19,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:46:19,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb4753a1e2bfb1b3346a60 HTTP/1.1" 200 None 2026-03-21 00:46:19,158 - lava_callback - ERROR - Error processing callback for node 69bb4753a1e2bfb1b3346a60: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-21 00:46:19,640 - lava_callback - INFO - Received callback for node 69bb4756a1e2bfb1b3346a6b INFO: 172.21.0.1:45496 - "POST /node/69bb4756a1e2bfb1b3346a6b HTTP/1.0" 202 Accepted 2026-03-21 00:46:19,649 - lava_callback - INFO - Starting async job submit for node 69bb4756a1e2bfb1b3346a6b 2026-03-21 00:46:19,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:46:19,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4756a1e2bfb1b3346a6b HTTP/1.1" 200 None 2026-03-21 00:46:19,682 - lava_callback - DEBUG - Processing node 69bb4756a1e2bfb1b3346a6b: baseline-arm-baylibre 2026-03-21 00:46:19,683 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb4756a1e2bfb1b3346a6b 2026-03-21 00:46:19,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:46:19,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4756a1e2bfb1b3346a6b HTTP/1.1" 200 None 2026-03-21 00:46:19,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:46:19,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-21 00:46:19,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:46:19,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb4756a1e2bfb1b3346a6b HTTP/1.1" 200 None 2026-03-21 00:46:19,776 - lava_callback - ERROR - Error processing callback for node 69bb4756a1e2bfb1b3346a6b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:32980 - "GET / HTTP/1.0" 200 OK 2026-03-21 00:46:30,835 - lava_callback - INFO - Received callback for node 69bb475fa1e2bfb1b3346a87 INFO: 172.21.0.1:59650 - "POST /node/69bb475fa1e2bfb1b3346a87 HTTP/1.0" 202 Accepted 2026-03-21 00:46:30,844 - lava_callback - INFO - Starting async job submit for node 69bb475fa1e2bfb1b3346a87 2026-03-21 00:46:30,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:46:30,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb475fa1e2bfb1b3346a87 HTTP/1.1" 200 None 2026-03-21 00:46:30,877 - lava_callback - DEBUG - Processing node 69bb475fa1e2bfb1b3346a87: baseline-arm-clabbe 2026-03-21 00:46:30,877 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb475fa1e2bfb1b3346a87 2026-03-21 00:46:30,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:46:30,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb475fa1e2bfb1b3346a87 HTTP/1.1" 200 None 2026-03-21 00:46:30,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:46:30,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-21 00:46:30,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:46:30,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb475fa1e2bfb1b3346a87 HTTP/1.1" 200 None 2026-03-21 00:46:30,977 - lava_callback - ERROR - Error processing callback for node 69bb475fa1e2bfb1b3346a87: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-21 00:46:31,467 - lava_callback - INFO - Received callback for node 69bb4764a1e2bfb1b3346a96 INFO: 172.21.0.1:59666 - "POST /node/69bb4764a1e2bfb1b3346a96 HTTP/1.0" 202 Accepted 2026-03-21 00:46:31,473 - lava_callback - INFO - Starting async job submit for node 69bb4764a1e2bfb1b3346a96 2026-03-21 00:46:31,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:46:31,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4764a1e2bfb1b3346a96 HTTP/1.1" 200 None 2026-03-21 00:46:31,489 - lava_callback - DEBUG - Processing node 69bb4764a1e2bfb1b3346a96: baseline-arm-clabbe 2026-03-21 00:46:31,489 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb4764a1e2bfb1b3346a96 2026-03-21 00:46:31,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:46:31,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4764a1e2bfb1b3346a96 HTTP/1.1" 200 None 2026-03-21 00:46:31,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:46:31,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-21 00:46:31,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:46:31,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb4764a1e2bfb1b3346a96 HTTP/1.1" 200 None 2026-03-21 00:46:31,542 - lava_callback - ERROR - Error processing callback for node 69bb4764a1e2bfb1b3346a96: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-21 00:46:32,111 - lava_callback - INFO - Received callback for node 69bb4765a1e2bfb1b3346a99 INFO: 172.21.0.1:59672 - "POST /node/69bb4765a1e2bfb1b3346a99 HTTP/1.0" 202 Accepted 2026-03-21 00:46:32,117 - lava_callback - INFO - Starting async job submit for node 69bb4765a1e2bfb1b3346a99 2026-03-21 00:46:32,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:46:32,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4765a1e2bfb1b3346a99 HTTP/1.1" 200 None 2026-03-21 00:46:32,133 - lava_callback - DEBUG - Processing node 69bb4765a1e2bfb1b3346a99: baseline-arm-clabbe 2026-03-21 00:46:32,133 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb4765a1e2bfb1b3346a99 2026-03-21 00:46:32,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:46:32,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4765a1e2bfb1b3346a99 HTTP/1.1" 200 None 2026-03-21 00:46:32,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:46:32,170 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-21 00:46:32,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:46:32,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb4765a1e2bfb1b3346a99 HTTP/1.1" 200 None 2026-03-21 00:46:32,193 - lava_callback - ERROR - Error processing callback for node 69bb4765a1e2bfb1b3346a99: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-21 00:46:33,014 - lava_callback - INFO - Received callback for node 69bde8bed54b5122738c645e INFO: 172.21.0.1:59674 - "POST /node/69bde8bed54b5122738c645e HTTP/1.0" 202 Accepted 2026-03-21 00:46:33,020 - lava_callback - INFO - Starting async job submit for node 69bde8bed54b5122738c645e 2026-03-21 00:46:33,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:46:33,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde8bed54b5122738c645e HTTP/1.1" 200 None 2026-03-21 00:46:33,061 - lava_callback - DEBUG - Processing node 69bde8bed54b5122738c645e: baseline-arm-clabbe 2026-03-21 00:46:33,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:46:33,588 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:46:33,589 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69bde8bed54b5122738c645e/log.txt.gz 2026-03-21 00:46:33,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:46:34,176 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:46:34,178 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69bde8bed54b5122738c645e/lava-logs.yaml 2026-03-21 00:46:34,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:46:34,323 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:46:34,325 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69bde8bed54b5122738c645e/lava_callback.json.gz 2026-03-21 00:46:34,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:46:34,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde8bed54b5122738c645e HTTP/1.1" 200 None 2026-03-21 00:46:34,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:46:34,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-21 00:46:34,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:46:34,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bde8bed54b5122738c645e HTTP/1.1" 200 None 2026-03-21 00:46:34,433 - lava_callback - INFO - Completed processing callback for node 69bde8bed54b5122738c645e 2026-03-21 00:46:35,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:46:35,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:59678 - "GET / HTTP/1.0" 200 OK 2026-03-21 00:46:36,895 - lava_callback - INFO - Received callback for node 69bb4757a1e2bfb1b3346a6e INFO: 172.21.0.1:59690 - "POST /node/69bb4757a1e2bfb1b3346a6e HTTP/1.0" 202 Accepted 2026-03-21 00:46:36,904 - lava_callback - INFO - Starting async job submit for node 69bb4757a1e2bfb1b3346a6e 2026-03-21 00:46:36,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:46:36,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4757a1e2bfb1b3346a6e HTTP/1.1" 200 None 2026-03-21 00:46:36,943 - lava_callback - DEBUG - Processing node 69bb4757a1e2bfb1b3346a6e: baseline-arm-baylibre 2026-03-21 00:46:36,943 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb4757a1e2bfb1b3346a6e 2026-03-21 00:46:36,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:46:36,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4757a1e2bfb1b3346a6e HTTP/1.1" 200 None 2026-03-21 00:46:36,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:46:36,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-21 00:46:36,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:46:37,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb4757a1e2bfb1b3346a6e HTTP/1.1" 200 None 2026-03-21 00:46:37,012 - lava_callback - ERROR - Error processing callback for node 69bb4757a1e2bfb1b3346a6e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41670 - "GET / HTTP/1.0" 200 OK 2026-03-21 00:46:50,910 - lava_callback - INFO - Received callback for node 69bb4766a1e2bfb1b3346a9c INFO: 172.21.0.1:44066 - "POST /node/69bb4766a1e2bfb1b3346a9c HTTP/1.0" 202 Accepted 2026-03-21 00:46:50,916 - lava_callback - INFO - Starting async job submit for node 69bb4766a1e2bfb1b3346a9c 2026-03-21 00:46:50,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:46:50,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4766a1e2bfb1b3346a9c HTTP/1.1" 200 None 2026-03-21 00:46:50,939 - lava_callback - DEBUG - Processing node 69bb4766a1e2bfb1b3346a9c: baseline-arm-clabbe 2026-03-21 00:46:50,939 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb4766a1e2bfb1b3346a9c 2026-03-21 00:46:50,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:46:50,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4766a1e2bfb1b3346a9c HTTP/1.1" 200 None 2026-03-21 00:46:50,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:46:50,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-21 00:46:50,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:46:50,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb4766a1e2bfb1b3346a9c HTTP/1.1" 200 None 2026-03-21 00:46:51,000 - lava_callback - ERROR - Error processing callback for node 69bb4766a1e2bfb1b3346a9c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-21 00:46:51,573 - lava_callback - INFO - Received callback for node 69bb4768a1e2bfb1b3346aca INFO: 172.21.0.1:44074 - "POST /node/69bb4768a1e2bfb1b3346aca HTTP/1.0" 202 Accepted 2026-03-21 00:46:51,579 - lava_callback - INFO - Starting async job submit for node 69bb4768a1e2bfb1b3346aca 2026-03-21 00:46:51,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:46:51,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4768a1e2bfb1b3346aca HTTP/1.1" 200 None 2026-03-21 00:46:51,595 - lava_callback - DEBUG - Processing node 69bb4768a1e2bfb1b3346aca: baseline-arm-clabbe 2026-03-21 00:46:51,595 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb4768a1e2bfb1b3346aca 2026-03-21 00:46:51,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:46:51,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4768a1e2bfb1b3346aca HTTP/1.1" 200 None 2026-03-21 00:46:51,611 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:46:51,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-21 00:46:51,625 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:46:51,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb4768a1e2bfb1b3346aca HTTP/1.1" 200 None 2026-03-21 00:46:51,650 - lava_callback - ERROR - Error processing callback for node 69bb4768a1e2bfb1b3346aca: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34706 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59344 - "GET / HTTP/1.0" 200 OK 2026-03-21 00:47:50,736 - lava_callback - INFO - Received callback for node 69bde618d54b5122738c6391 INFO: 172.21.0.1:37590 - "POST /node/69bde618d54b5122738c6391 HTTP/1.0" 202 Accepted 2026-03-21 00:47:50,743 - lava_callback - INFO - Starting async job submit for node 69bde618d54b5122738c6391 2026-03-21 00:47:50,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:47:50,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde618d54b5122738c6391 HTTP/1.1" 200 None 2026-03-21 00:47:50,837 - lava_callback - DEBUG - Processing node 69bde618d54b5122738c6391: ltp-smoke 2026-03-21 00:47:51,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:47:51,451 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:47:51,453 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69bde618d54b5122738c6391/log.txt.gz 2026-03-21 00:47:51,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:47:51,954 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:47:51,956 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69bde618d54b5122738c6391/lava-logs.yaml 2026-03-21 00:47:51,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:47:52,183 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:47:52,185 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69bde618d54b5122738c6391/lava_callback.json.gz 2026-03-21 00:47:52,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:47:52,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde618d54b5122738c6391 HTTP/1.1" 200 None 2026-03-21 00:47:52,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:47:52,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde35fd54b5122738c3d99 HTTP/1.1" 200 None 2026-03-21 00:47:52,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:47:52,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bde618d54b5122738c6391 HTTP/1.1" 200 None 2026-03-21 00:47:52,531 - lava_callback - INFO - Completed processing callback for node 69bde618d54b5122738c6391 INFO: 172.21.0.1:37604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51530 - "GET / HTTP/1.0" 200 OK 2026-03-21 00:48:05,346 - lava_callback - INFO - Received callback for node 69bdea83d54b5122738c6626 INFO: 172.21.0.1:51536 - "POST /node/69bdea83d54b5122738c6626 HTTP/1.0" 202 Accepted 2026-03-21 00:48:05,353 - lava_callback - INFO - Starting async job submit for node 69bdea83d54b5122738c6626 2026-03-21 00:48:05,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:48:05,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdea83d54b5122738c6626 HTTP/1.1" 200 None 2026-03-21 00:48:05,393 - lava_callback - DEBUG - Processing node 69bdea83d54b5122738c6626: baseline-arm-pengutronix 2026-03-21 00:48:05,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:48:05,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:48:05,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-21 00:48:05,675 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:48:05,677 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69bdea83d54b5122738c6626/log.txt.gz 2026-03-21 00:48:05,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:48:05,957 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:48:05,959 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69bdea83d54b5122738c6626/lava-logs.yaml 2026-03-21 00:48:05,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:48:06,114 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:48:06,115 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69bdea83d54b5122738c6626/lava_callback.json.gz 2026-03-21 00:48:06,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:48:06,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdea83d54b5122738c6626 HTTP/1.1" 200 None 2026-03-21 00:48:06,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:48:06,164 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-21 00:48:06,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:48:06,190 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdea83d54b5122738c6626 HTTP/1.1" 200 None 2026-03-21 00:48:06,191 - lava_callback - INFO - Completed processing callback for node 69bdea83d54b5122738c6626 INFO: 172.21.0.1:51548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51808 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50382 - "GET / HTTP/1.0" 200 OK 2026-03-21 00:48:35,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:48:35,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:33902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60678 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39286 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-21 00:50:16,330 - lava_callback - INFO - Received callback for node 69bdeb33d54b5122738c666d INFO: 172.21.0.1:39298 - "POST /node/69bdeb33d54b5122738c666d HTTP/1.0" 202 Accepted 2026-03-21 00:50:16,339 - lava_callback - INFO - Starting async job submit for node 69bdeb33d54b5122738c666d 2026-03-21 00:50:16,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:50:16,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdeb33d54b5122738c666d HTTP/1.1" 200 None 2026-03-21 00:50:16,376 - lava_callback - DEBUG - Processing node 69bdeb33d54b5122738c666d: baseline-arm 2026-03-21 00:50:16,384 - lava_callback - WARNING - Failed to upload log 2026-03-21 00:50:16,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:39314 - "GET / HTTP/1.0" 200 OK 2026-03-21 00:50:16,723 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:50:16,725 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69bdeb33d54b5122738c666d/lava-logs.yaml 2026-03-21 00:50:16,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:50:17,025 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:50:17,027 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69bdeb33d54b5122738c666d/lava_callback.json.gz 2026-03-21 00:50:17,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:50:17,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdeb33d54b5122738c666d HTTP/1.1" 200 None 2026-03-21 00:50:17,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:50:17,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-21 00:50:17,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:50:17,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdeb33d54b5122738c666d HTTP/1.1" 200 None 2026-03-21 00:50:17,138 - lava_callback - INFO - Completed processing callback for node 69bdeb33d54b5122738c666d INFO: 172.21.0.1:38534 - "GET / HTTP/1.0" 200 OK 2026-03-21 00:50:28,727 - lava_callback - INFO - Received callback for node 69bd7ae60e93e2314723e767 INFO: 172.21.0.1:38538 - "POST /node/69bd7ae60e93e2314723e767 HTTP/1.0" 202 Accepted 2026-03-21 00:50:28,737 - lava_callback - INFO - Starting async job submit for node 69bd7ae60e93e2314723e767 2026-03-21 00:50:28,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:50:28,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd7ae60e93e2314723e767 HTTP/1.1" 200 None 2026-03-21 00:50:28,808 - lava_callback - DEBUG - Processing node 69bd7ae60e93e2314723e767: baseline-arm64-broonie 2026-03-21 00:50:29,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:50:29,540 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:50:29,542 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69bd7ae60e93e2314723e767/log.txt.gz 2026-03-21 00:50:29,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:50:30,274 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:50:30,276 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69bd7ae60e93e2314723e767/lava-logs.yaml 2026-03-21 00:50:30,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:50:30,641 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:50:30,643 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69bd7ae60e93e2314723e767/lava_callback.json.gz 2026-03-21 00:50:30,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:50:30,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd7ae60e93e2314723e767 HTTP/1.1" 200 None 2026-03-21 00:50:30,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:50:30,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-21 00:50:30,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:50:30,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd7ae60e93e2314723e767 HTTP/1.1" 200 None 2026-03-21 00:50:30,818 - lava_callback - INFO - Completed processing callback for node 69bd7ae60e93e2314723e767 2026-03-21 00:50:35,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:50:35,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:36252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60052 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53114 - "GET / HTTP/1.0" 200 OK 2026-03-21 00:51:48,579 - lava_callback - INFO - Received callback for node 69bdd66a0e93e23147242cff INFO: 172.21.0.1:53124 - "POST /node/69bdd66a0e93e23147242cff HTTP/1.0" 202 Accepted 2026-03-21 00:51:48,585 - lava_callback - INFO - Starting async job submit for node 69bdd66a0e93e23147242cff 2026-03-21 00:51:48,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:51:48,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd66a0e93e23147242cff HTTP/1.1" 200 None 2026-03-21 00:51:48,625 - lava_callback - DEBUG - Processing node 69bdd66a0e93e23147242cff: baseline-arm64-broonie 2026-03-21 00:51:48,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:51:49,019 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:51:49,021 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69bdd66a0e93e23147242cff/log.txt.gz 2026-03-21 00:51:49,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:51:49,364 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:51:49,366 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69bdd66a0e93e23147242cff/lava-logs.yaml 2026-03-21 00:51:49,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:51:49,534 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:51:49,536 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69bdd66a0e93e23147242cff/lava_callback.json.gz 2026-03-21 00:51:49,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:51:49,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd66a0e93e23147242cff HTTP/1.1" 200 None 2026-03-21 00:51:49,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:51:49,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-21 00:51:49,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:51:49,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdd66a0e93e23147242cff HTTP/1.1" 200 None 2026-03-21 00:51:49,621 - lava_callback - INFO - Completed processing callback for node 69bdd66a0e93e23147242cff INFO: 172.21.0.1:34468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60658 - "GET / HTTP/1.0" 200 OK 2026-03-21 00:52:05,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:52:05,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:60668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56664 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49696 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49698 - "GET / HTTP/1.0" 200 OK 2026-03-21 00:53:17,399 - lava_callback - INFO - Received callback for node 69bb48faa1e2bfb1b3346c73 INFO: 172.21.0.1:49700 - "POST /node/69bb48faa1e2bfb1b3346c73 HTTP/1.0" 202 Accepted 2026-03-21 00:53:17,405 - lava_callback - INFO - Starting async job submit for node 69bb48faa1e2bfb1b3346c73 2026-03-21 00:53:17,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:53:17,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb48faa1e2bfb1b3346c73 HTTP/1.1" 200 None 2026-03-21 00:53:17,420 - lava_callback - DEBUG - Processing node 69bb48faa1e2bfb1b3346c73: baseline-arm-baylibre 2026-03-21 00:53:17,420 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb48faa1e2bfb1b3346c73 2026-03-21 00:53:17,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:53:17,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb48faa1e2bfb1b3346c73 HTTP/1.1" 200 None 2026-03-21 00:53:17,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:53:17,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-21 00:53:17,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:53:17,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb48faa1e2bfb1b3346c73 HTTP/1.1" 200 None 2026-03-21 00:53:17,475 - lava_callback - ERROR - Error processing callback for node 69bb48faa1e2bfb1b3346c73: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-21 00:53:19,057 - lava_callback - INFO - Received callback for node 69bde8b1d54b5122738c640e INFO: 172.21.0.1:49714 - "POST /node/69bde8b1d54b5122738c640e HTTP/1.0" 202 Accepted 2026-03-21 00:53:19,063 - lava_callback - INFO - Starting async job submit for node 69bde8b1d54b5122738c640e 2026-03-21 00:53:19,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:53:19,110 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde8b1d54b5122738c640e HTTP/1.1" 200 None 2026-03-21 00:53:19,111 - lava_callback - DEBUG - Processing node 69bde8b1d54b5122738c640e: baseline-arm-broonie 2026-03-21 00:53:19,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:53:19,457 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:53:19,459 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69bde8b1d54b5122738c640e/log.txt.gz 2026-03-21 00:53:19,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:53:19,781 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:53:19,782 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69bde8b1d54b5122738c640e/lava-logs.yaml 2026-03-21 00:53:19,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:53:19,958 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:53:19,960 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69bde8b1d54b5122738c640e/lava_callback.json.gz 2026-03-21 00:53:19,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:53:19,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde8b1d54b5122738c640e HTTP/1.1" 200 None 2026-03-21 00:53:19,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:53:20,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-21 00:53:20,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:53:20,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bde8b1d54b5122738c640e HTTP/1.1" 200 None 2026-03-21 00:53:20,047 - lava_callback - INFO - Completed processing callback for node 69bde8b1d54b5122738c640e INFO: 172.21.0.1:55118 - "GET / HTTP/1.0" 200 OK 2026-03-21 00:53:35,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:53:35,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:44456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47358 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55044 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55060 - "GET / HTTP/1.0" 200 OK 2026-03-21 00:55:17,401 - lava_callback - INFO - Received callback for node 69bb4966a1e2bfb1b3346ca5 INFO: 172.21.0.1:55066 - "POST /node/69bb4966a1e2bfb1b3346ca5 HTTP/1.0" 202 Accepted 2026-03-21 00:55:17,406 - lava_callback - INFO - Starting async job submit for node 69bb4966a1e2bfb1b3346ca5 2026-03-21 00:55:17,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:55:17,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4966a1e2bfb1b3346ca5 HTTP/1.1" 200 None 2026-03-21 00:55:17,422 - lava_callback - DEBUG - Processing node 69bb4966a1e2bfb1b3346ca5: baseline-arm-baylibre 2026-03-21 00:55:17,422 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb4966a1e2bfb1b3346ca5 2026-03-21 00:55:17,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:55:17,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4966a1e2bfb1b3346ca5 HTTP/1.1" 200 None 2026-03-21 00:55:17,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:55:17,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-21 00:55:17,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:55:17,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb4966a1e2bfb1b3346ca5 HTTP/1.1" 200 None 2026-03-21 00:55:17,486 - lava_callback - ERROR - Error processing callback for node 69bb4966a1e2bfb1b3346ca5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52228 - "GET / HTTP/1.0" 200 OK 2026-03-21 00:55:57,387 - lava_callback - INFO - Received callback for node 69bb498ba1e2bfb1b3346cb7 INFO: 172.21.0.1:52240 - "POST /node/69bb498ba1e2bfb1b3346cb7 HTTP/1.0" 202 Accepted 2026-03-21 00:55:57,392 - lava_callback - INFO - Starting async job submit for node 69bb498ba1e2bfb1b3346cb7 2026-03-21 00:55:57,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:55:57,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb498ba1e2bfb1b3346cb7 HTTP/1.1" 200 None 2026-03-21 00:55:57,408 - lava_callback - DEBUG - Processing node 69bb498ba1e2bfb1b3346cb7: baseline-arm-baylibre 2026-03-21 00:55:57,408 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb498ba1e2bfb1b3346cb7 2026-03-21 00:55:57,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:55:57,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb498ba1e2bfb1b3346cb7 HTTP/1.1" 200 None 2026-03-21 00:55:57,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:55:57,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89da7fbe29c53ae85cce2 HTTP/1.1" 200 None 2026-03-21 00:55:57,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:55:57,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb498ba1e2bfb1b3346cb7 HTTP/1.1" 200 None 2026-03-21 00:55:57,493 - lava_callback - ERROR - Error processing callback for node 69bb498ba1e2bfb1b3346cb7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54284 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54296 - "GET / HTTP/1.0" 200 OK 2026-03-21 00:56:17,412 - lava_callback - INFO - Received callback for node 69bb49afa1e2bfb1b3346cc9 INFO: 172.21.0.1:54302 - "POST /node/69bb49afa1e2bfb1b3346cc9 HTTP/1.0" 202 Accepted 2026-03-21 00:56:17,418 - lava_callback - INFO - Starting async job submit for node 69bb49afa1e2bfb1b3346cc9 2026-03-21 00:56:17,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:56:17,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb49afa1e2bfb1b3346cc9 HTTP/1.1" 200 None 2026-03-21 00:56:17,434 - lava_callback - DEBUG - Processing node 69bb49afa1e2bfb1b3346cc9: baseline-arm-baylibre 2026-03-21 00:56:17,434 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb49afa1e2bfb1b3346cc9 2026-03-21 00:56:17,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:56:17,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb49afa1e2bfb1b3346cc9 HTTP/1.1" 200 None 2026-03-21 00:56:17,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:56:17,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89da7fbe29c53ae85cce2 HTTP/1.1" 200 None 2026-03-21 00:56:17,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:56:17,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb49afa1e2bfb1b3346cc9 HTTP/1.1" 200 None 2026-03-21 00:56:17,503 - lava_callback - ERROR - Error processing callback for node 69bb49afa1e2bfb1b3346cc9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:32822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40456 - "GET / HTTP/1.0" 200 OK 2026-03-21 00:57:06,547 - lava_callback - INFO - Received callback for node 69bde8a8d54b5122738c63fd INFO: 172.21.0.1:40468 - "POST /node/69bde8a8d54b5122738c63fd HTTP/1.0" 202 Accepted 2026-03-21 00:57:06,559 - lava_callback - INFO - Starting async job submit for node 69bde8a8d54b5122738c63fd 2026-03-21 00:57:06,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:57:06,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde8a8d54b5122738c63fd HTTP/1.1" 200 None 2026-03-21 00:57:06,686 - lava_callback - DEBUG - Processing node 69bde8a8d54b5122738c63fd: baseline-arm 2026-03-21 00:57:07,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:57:08,148 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:57:08,150 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69bde8a8d54b5122738c63fd/log.txt.gz 2026-03-21 00:57:08,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:57:09,296 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:57:09,299 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69bde8a8d54b5122738c63fd/lava-logs.yaml 2026-03-21 00:57:09,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:57:09,606 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:57:09,608 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69bde8a8d54b5122738c63fd/lava_callback.json.gz DEBUG: Setup failed for 69bde8a8d54b5122738c63fd. Transitting job node result: incomplete -> fail 2026-03-21 00:57:09,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:57:09,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde8a8d54b5122738c63fd HTTP/1.1" 200 None 2026-03-21 00:57:09,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:57:09,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-21 00:57:09,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:57:09,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bde8a8d54b5122738c63fd HTTP/1.1" 200 None 2026-03-21 00:57:09,916 - lava_callback - INFO - Completed processing callback for node 69bde8a8d54b5122738c63fd INFO: 172.21.0.1:34050 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34052 - "GET / HTTP/1.0" 200 OK 2026-03-21 00:57:19,125 - lava_callback - INFO - Received callback for node 69bde8d8d54b5122738c649e INFO: 172.21.0.1:34068 - "POST /node/69bde8d8d54b5122738c649e HTTP/1.0" 202 Accepted 2026-03-21 00:57:19,136 - lava_callback - INFO - Starting async job submit for node 69bde8d8d54b5122738c649e 2026-03-21 00:57:19,163 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:57:19,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde8d8d54b5122738c649e HTTP/1.1" 200 None 2026-03-21 00:57:19,198 - lava_callback - DEBUG - Processing node 69bde8d8d54b5122738c649e: ltp-cap_bounds 2026-03-21 00:57:19,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:57:20,292 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:57:20,294 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69bde8d8d54b5122738c649e/log.txt.gz 2026-03-21 00:57:20,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:57:20,739 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:57:20,741 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69bde8d8d54b5122738c649e/lava-logs.yaml 2026-03-21 00:57:20,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:57:20,920 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:57:20,921 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69bde8d8d54b5122738c649e/lava_callback.json.gz 2026-03-21 00:57:20,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:57:20,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde8d8d54b5122738c649e HTTP/1.1" 200 None 2026-03-21 00:57:20,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:57:20,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-21 00:57:20,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:57:21,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bde8d8d54b5122738c649e HTTP/1.1" 200 None 2026-03-21 00:57:21,028 - lava_callback - INFO - Completed processing callback for node 69bde8d8d54b5122738c649e INFO: 172.21.0.1:59116 - "GET / HTTP/1.0" 200 OK 2026-03-21 00:57:35,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:57:35,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:51496 - "GET / HTTP/1.0" 200 OK 2026-03-21 00:57:37,532 - lava_callback - INFO - Received callback for node 69bb49f9a1e2bfb1b3346cec INFO: 172.21.0.1:51504 - "POST /node/69bb49f9a1e2bfb1b3346cec HTTP/1.0" 202 Accepted 2026-03-21 00:57:37,541 - lava_callback - INFO - Starting async job submit for node 69bb49f9a1e2bfb1b3346cec 2026-03-21 00:57:37,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:57:37,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb49f9a1e2bfb1b3346cec HTTP/1.1" 200 None 2026-03-21 00:57:37,564 - lava_callback - DEBUG - Processing node 69bb49f9a1e2bfb1b3346cec: baseline-arm-baylibre 2026-03-21 00:57:37,565 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb49f9a1e2bfb1b3346cec 2026-03-21 00:57:37,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:57:37,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb49f9a1e2bfb1b3346cec HTTP/1.1" 200 None 2026-03-21 00:57:37,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:57:37,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89da7fbe29c53ae85cce2 HTTP/1.1" 200 None 2026-03-21 00:57:37,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:57:37,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb49f9a1e2bfb1b3346cec HTTP/1.1" 200 None 2026-03-21 00:57:37,653 - lava_callback - ERROR - Error processing callback for node 69bb49f9a1e2bfb1b3346cec: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-21 00:57:40,743 - lava_callback - INFO - Received callback for node 69bde8d9d54b5122738c64a1 INFO: 172.21.0.1:33222 - "POST /node/69bde8d9d54b5122738c64a1 HTTP/1.0" 202 Accepted 2026-03-21 00:57:40,754 - lava_callback - INFO - Starting async job submit for node 69bde8d9d54b5122738c64a1 2026-03-21 00:57:40,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:57:40,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde8d9d54b5122738c64a1 HTTP/1.1" 200 None 2026-03-21 00:57:40,797 - lava_callback - DEBUG - Processing node 69bde8d9d54b5122738c64a1: ltp-containers 2026-03-21 00:57:41,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:57:41,797 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:57:41,799 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-containers-69bde8d9d54b5122738c64a1/log.txt.gz 2026-03-21 00:57:42,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:57:42,249 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:57:42,251 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-containers-69bde8d9d54b5122738c64a1/lava-logs.yaml 2026-03-21 00:57:42,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:57:42,414 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:57:42,417 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-containers-69bde8d9d54b5122738c64a1/lava_callback.json.gz 2026-03-21 00:57:42,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:57:42,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde8d9d54b5122738c64a1 HTTP/1.1" 200 None 2026-03-21 00:57:42,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:57:42,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-21 00:57:42,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:57:42,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bde8d9d54b5122738c64a1 HTTP/1.1" 200 None 2026-03-21 00:57:42,573 - lava_callback - INFO - Completed processing callback for node 69bde8d9d54b5122738c64a1 INFO: 172.21.0.1:33230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38284 - "GET / HTTP/1.0" 200 OK 2026-03-21 00:58:05,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:58:05,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:38286 - "GET / HTTP/1.0" 200 OK 2026-03-21 00:58:08,915 - lava_callback - INFO - Received callback for node 69bde8dbd54b5122738c64a5 INFO: 172.21.0.1:38292 - "POST /node/69bde8dbd54b5122738c64a5 HTTP/1.0" 202 Accepted 2026-03-21 00:58:08,922 - lava_callback - INFO - Starting async job submit for node 69bde8dbd54b5122738c64a5 2026-03-21 00:58:08,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:58:08,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde8dbd54b5122738c64a5 HTTP/1.1" 200 None 2026-03-21 00:58:08,971 - lava_callback - DEBUG - Processing node 69bde8dbd54b5122738c64a5: ltp-controllers 2026-03-21 00:58:09,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:58:09,963 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:58:09,965 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-controllers-69bde8dbd54b5122738c64a5/log.txt.gz 2026-03-21 00:58:10,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:58:10,489 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:58:10,491 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-controllers-69bde8dbd54b5122738c64a5/lava-logs.yaml 2026-03-21 00:58:10,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:58:11,397 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:58:11,400 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-controllers-69bde8dbd54b5122738c64a5/lava_callback.json.gz 2026-03-21 00:58:11,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:58:11,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde8dbd54b5122738c64a5 HTTP/1.1" 200 None 2026-03-21 00:58:11,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:58:11,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-21 00:58:11,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:58:11,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bde8dbd54b5122738c64a5 HTTP/1.1" 200 None 2026-03-21 00:58:11,499 - lava_callback - INFO - Completed processing callback for node 69bde8dbd54b5122738c64a5 INFO: 172.21.0.1:42546 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42560 - "GET / HTTP/1.0" 200 OK 2026-03-21 00:58:17,499 - lava_callback - INFO - Received callback for node 69bb4a1ca1e2bfb1b3346cfa INFO: 172.21.0.1:42572 - "POST /node/69bb4a1ca1e2bfb1b3346cfa HTTP/1.0" 202 Accepted 2026-03-21 00:58:17,505 - lava_callback - INFO - Starting async job submit for node 69bb4a1ca1e2bfb1b3346cfa 2026-03-21 00:58:17,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:58:17,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4a1ca1e2bfb1b3346cfa HTTP/1.1" 200 None 2026-03-21 00:58:17,524 - lava_callback - DEBUG - Processing node 69bb4a1ca1e2bfb1b3346cfa: baseline-arm-baylibre 2026-03-21 00:58:17,525 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb4a1ca1e2bfb1b3346cfa 2026-03-21 00:58:17,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:58:17,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4a1ca1e2bfb1b3346cfa HTTP/1.1" 200 None 2026-03-21 00:58:17,548 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:58:17,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89da7fbe29c53ae85cce2 HTTP/1.1" 200 None 2026-03-21 00:58:17,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:58:17,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb4a1ca1e2bfb1b3346cfa HTTP/1.1" 200 None 2026-03-21 00:58:17,601 - lava_callback - ERROR - Error processing callback for node 69bb4a1ca1e2bfb1b3346cfa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49176 - "GET / HTTP/1.0" 200 OK 2026-03-21 00:58:35,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:58:35,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:37712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34036 - "GET / HTTP/1.0" 200 OK 2026-03-21 00:58:57,581 - lava_callback - INFO - Received callback for node 69bb4a41a1e2bfb1b3346d07 INFO: 172.21.0.1:34042 - "POST /node/69bb4a41a1e2bfb1b3346d07 HTTP/1.0" 202 Accepted 2026-03-21 00:58:57,589 - lava_callback - INFO - Starting async job submit for node 69bb4a41a1e2bfb1b3346d07 2026-03-21 00:58:57,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:58:57,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4a41a1e2bfb1b3346d07 HTTP/1.1" 200 None 2026-03-21 00:58:57,619 - lava_callback - DEBUG - Processing node 69bb4a41a1e2bfb1b3346d07: baseline-arm-baylibre 2026-03-21 00:58:57,619 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb4a41a1e2bfb1b3346d07 2026-03-21 00:58:57,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:58:57,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4a41a1e2bfb1b3346d07 HTTP/1.1" 200 None 2026-03-21 00:58:57,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:58:57,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89da7fbe29c53ae85cce2 HTTP/1.1" 200 None 2026-03-21 00:58:57,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:58:57,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb4a41a1e2bfb1b3346d07 HTTP/1.1" 200 None 2026-03-21 00:58:57,721 - lava_callback - ERROR - Error processing callback for node 69bb4a41a1e2bfb1b3346d07: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42362 - "GET / HTTP/1.0" 200 OK 2026-03-21 00:59:13,633 - lava_callback - INFO - Received callback for node 69bde8c5d54b5122738c6472 INFO: 172.21.0.1:50910 - "POST /node/69bde8c5d54b5122738c6472 HTTP/1.0" 202 Accepted 2026-03-21 00:59:13,685 - lava_callback - INFO - Starting async job submit for node 69bde8c5d54b5122738c6472 2026-03-21 00:59:13,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:59:13,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde8c5d54b5122738c6472 HTTP/1.1" 200 None 2026-03-21 00:59:13,741 - lava_callback - DEBUG - Processing node 69bde8c5d54b5122738c6472: baseline-arm-pengutronix 2026-03-21 00:59:13,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:59:13,999 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:59:14,000 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69bde8c5d54b5122738c6472/log.txt.gz 2026-03-21 00:59:14,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:59:14,308 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:59:14,310 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69bde8c5d54b5122738c6472/lava-logs.yaml 2026-03-21 00:59:14,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:59:14,535 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:59:14,537 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69bde8c5d54b5122738c6472/lava_callback.json.gz DEBUG: Setup failed for 69bde8c5d54b5122738c6472. Transitting job node result: incomplete -> fail 2026-03-21 00:59:14,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:59:14,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde8c5d54b5122738c6472 HTTP/1.1" 200 None 2026-03-21 00:59:14,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:59:14,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-21 00:59:14,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:59:14,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bde8c5d54b5122738c6472 HTTP/1.1" 200 None 2026-03-21 00:59:14,895 - lava_callback - INFO - Completed processing callback for node 69bde8c5d54b5122738c6472 INFO: 172.21.0.1:50920 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56090 - "GET / HTTP/1.0" 200 OK 2026-03-21 00:59:31,496 - lava_callback - INFO - Received callback for node 69bb4a67a1e2bfb1b3346d18 INFO: 172.21.0.1:50720 - "POST /node/69bb4a67a1e2bfb1b3346d18 HTTP/1.0" 202 Accepted 2026-03-21 00:59:31,502 - lava_callback - INFO - Starting async job submit for node 69bb4a67a1e2bfb1b3346d18 2026-03-21 00:59:31,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:59:31,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4a67a1e2bfb1b3346d18 HTTP/1.1" 200 None 2026-03-21 00:59:31,526 - lava_callback - DEBUG - Processing node 69bb4a67a1e2bfb1b3346d18: baseline-arm-clabbe 2026-03-21 00:59:31,526 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb4a67a1e2bfb1b3346d18 2026-03-21 00:59:31,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:59:31,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4a67a1e2bfb1b3346d18 HTTP/1.1" 200 None 2026-03-21 00:59:31,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:59:31,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89da7fbe29c53ae85cce2 HTTP/1.1" 200 None 2026-03-21 00:59:31,573 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:59:31,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb4a67a1e2bfb1b3346d18 HTTP/1.1" 200 None 2026-03-21 00:59:31,610 - lava_callback - ERROR - Error processing callback for node 69bb4a67a1e2bfb1b3346d18: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-21 00:59:35,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:59:35,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:50722 - "GET / HTTP/1.0" 200 OK 2026-03-21 00:59:40,522 - lava_callback - INFO - Received callback for node 69bde8ddd54b5122738c64b0 INFO: 172.21.0.1:38396 - "POST /node/69bde8ddd54b5122738c64b0 HTTP/1.0" 202 Accepted 2026-03-21 00:59:40,532 - lava_callback - INFO - Starting async job submit for node 69bde8ddd54b5122738c64b0 2026-03-21 00:59:40,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:59:40,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde8ddd54b5122738c64b0 HTTP/1.1" 200 None 2026-03-21 00:59:40,607 - lava_callback - DEBUG - Processing node 69bde8ddd54b5122738c64b0: ltp-cve 2026-03-21 00:59:41,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:59:41,341 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:59:41,343 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cve-69bde8ddd54b5122738c64b0/log.txt.gz 2026-03-21 00:59:41,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:59:42,410 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:59:42,413 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cve-69bde8ddd54b5122738c64b0/lava-logs.yaml 2026-03-21 00:59:42,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 00:59:42,628 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 00:59:42,630 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cve-69bde8ddd54b5122738c64b0/lava_callback.json.gz 2026-03-21 00:59:42,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:59:42,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde8ddd54b5122738c64b0 HTTP/1.1" 200 None 2026-03-21 00:59:42,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:59:42,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-21 00:59:42,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 00:59:42,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bde8ddd54b5122738c64b0 HTTP/1.1" 200 None 2026-03-21 00:59:42,716 - lava_callback - INFO - Completed processing callback for node 69bde8ddd54b5122738c64b0 INFO: 172.21.0.1:38402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50798 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:00:05,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:00:05,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:50802 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:00:11,795 - lava_callback - INFO - Received callback for node 69bb4a8aa1e2bfb1b3346d29 INFO: 172.21.0.1:47002 - "POST /node/69bb4a8aa1e2bfb1b3346d29 HTTP/1.0" 202 Accepted 2026-03-21 01:00:11,801 - lava_callback - INFO - Starting async job submit for node 69bb4a8aa1e2bfb1b3346d29 2026-03-21 01:00:11,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:00:11,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4a8aa1e2bfb1b3346d29 HTTP/1.1" 200 None 2026-03-21 01:00:11,819 - lava_callback - DEBUG - Processing node 69bb4a8aa1e2bfb1b3346d29: baseline-arm-clabbe 2026-03-21 01:00:11,820 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb4a8aa1e2bfb1b3346d29 2026-03-21 01:00:11,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:00:11,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4a8aa1e2bfb1b3346d29 HTTP/1.1" 200 None 2026-03-21 01:00:11,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:00:11,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89da7fbe29c53ae85cce2 HTTP/1.1" 200 None 2026-03-21 01:00:11,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:00:11,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb4a8aa1e2bfb1b3346d29 HTTP/1.1" 200 None 2026-03-21 01:00:11,901 - lava_callback - ERROR - Error processing callback for node 69bb4a8aa1e2bfb1b3346d29: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47012 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37670 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:00:37,810 - lava_callback - INFO - Received callback for node 69bb4aaca1e2bfb1b3346d3d INFO: 172.21.0.1:37672 - "POST /node/69bb4aaca1e2bfb1b3346d3d HTTP/1.0" 202 Accepted 2026-03-21 01:00:37,818 - lava_callback - INFO - Starting async job submit for node 69bb4aaca1e2bfb1b3346d3d 2026-03-21 01:00:37,819 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:00:37,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4aaca1e2bfb1b3346d3d HTTP/1.1" 200 None 2026-03-21 01:00:37,846 - lava_callback - DEBUG - Processing node 69bb4aaca1e2bfb1b3346d3d: baseline-arm-baylibre 2026-03-21 01:00:37,846 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb4aaca1e2bfb1b3346d3d 2026-03-21 01:00:37,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:00:37,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4aaca1e2bfb1b3346d3d HTTP/1.1" 200 None 2026-03-21 01:00:37,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:00:37,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89da7fbe29c53ae85cce2 HTTP/1.1" 200 None 2026-03-21 01:00:37,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:00:37,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb4aaca1e2bfb1b3346d3d HTTP/1.1" 200 None 2026-03-21 01:00:37,912 - lava_callback - ERROR - Error processing callback for node 69bb4aaca1e2bfb1b3346d3d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38988 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:01:11,535 - lava_callback - INFO - Received callback for node 69bb4ad2a1e2bfb1b3346d5e INFO: 172.21.0.1:59434 - "POST /node/69bb4ad2a1e2bfb1b3346d5e HTTP/1.0" 202 Accepted 2026-03-21 01:01:11,544 - lava_callback - INFO - Starting async job submit for node 69bb4ad2a1e2bfb1b3346d5e 2026-03-21 01:01:11,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:01:11,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4ad2a1e2bfb1b3346d5e HTTP/1.1" 200 None 2026-03-21 01:01:11,572 - lava_callback - DEBUG - Processing node 69bb4ad2a1e2bfb1b3346d5e: baseline-arm-clabbe 2026-03-21 01:01:11,572 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb4ad2a1e2bfb1b3346d5e 2026-03-21 01:01:11,573 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:01:11,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4ad2a1e2bfb1b3346d5e HTTP/1.1" 200 None 2026-03-21 01:01:11,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:01:11,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89da7fbe29c53ae85cce2 HTTP/1.1" 200 None 2026-03-21 01:01:11,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:01:11,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb4ad2a1e2bfb1b3346d5e HTTP/1.1" 200 None 2026-03-21 01:01:11,657 - lava_callback - ERROR - Error processing callback for node 69bb4ad2a1e2bfb1b3346d5e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59446 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59462 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:01:19,257 - lava_callback - INFO - Received callback for node 69bde8e0d54b5122738c64b6 INFO: 172.21.0.1:59464 - "POST /node/69bde8e0d54b5122738c64b6 HTTP/1.0" 202 Accepted 2026-03-21 01:01:19,266 - lava_callback - INFO - Starting async job submit for node 69bde8e0d54b5122738c64b6 2026-03-21 01:01:19,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:01:19,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde8e0d54b5122738c64b6 HTTP/1.1" 200 None 2026-03-21 01:01:19,340 - lava_callback - DEBUG - Processing node 69bde8e0d54b5122738c64b6: ltp-fs_bind 2026-03-21 01:01:19,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:01:20,190 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:01:20,192 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69bde8e0d54b5122738c64b6/log.txt.gz 2026-03-21 01:01:20,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:01:20,873 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:01:20,876 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69bde8e0d54b5122738c64b6/lava-logs.yaml 2026-03-21 01:01:20,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:01:21,139 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:01:21,141 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69bde8e0d54b5122738c64b6/lava_callback.json.gz 2026-03-21 01:01:21,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:01:21,195 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde8e0d54b5122738c64b6 HTTP/1.1" 200 None 2026-03-21 01:01:21,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:01:21,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-21 01:01:21,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:01:21,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bde8e0d54b5122738c64b6 HTTP/1.1" 200 None 2026-03-21 01:01:21,253 - lava_callback - INFO - Completed processing callback for node 69bde8e0d54b5122738c64b6 INFO: 172.21.0.1:36960 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:01:35,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:01:35,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:55814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40426 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:01:51,550 - lava_callback - INFO - Received callback for node 69bb4af6a1e2bfb1b3346d67 INFO: 172.21.0.1:59490 - "POST /node/69bb4af6a1e2bfb1b3346d67 HTTP/1.0" 202 Accepted 2026-03-21 01:01:51,562 - lava_callback - INFO - Starting async job submit for node 69bb4af6a1e2bfb1b3346d67 2026-03-21 01:01:51,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:01:51,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4af6a1e2bfb1b3346d67 HTTP/1.1" 200 None 2026-03-21 01:01:51,595 - lava_callback - DEBUG - Processing node 69bb4af6a1e2bfb1b3346d67: baseline-arm-clabbe 2026-03-21 01:01:51,596 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb4af6a1e2bfb1b3346d67 2026-03-21 01:01:51,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:01:51,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4af6a1e2bfb1b3346d67 HTTP/1.1" 200 None 2026-03-21 01:01:51,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:01:51,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-21 01:01:51,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:01:51,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb4af6a1e2bfb1b3346d67 HTTP/1.1" 200 None 2026-03-21 01:01:51,728 - lava_callback - ERROR - Error processing callback for node 69bb4af6a1e2bfb1b3346d67: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48638 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59908 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:02:31,607 - lava_callback - INFO - Received callback for node 69bb4b19a1e2bfb1b3346d71 INFO: 172.21.0.1:54630 - "POST /node/69bb4b19a1e2bfb1b3346d71 HTTP/1.0" 202 Accepted 2026-03-21 01:02:31,616 - lava_callback - INFO - Starting async job submit for node 69bb4b19a1e2bfb1b3346d71 2026-03-21 01:02:31,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:02:31,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4b19a1e2bfb1b3346d71 HTTP/1.1" 200 None 2026-03-21 01:02:31,642 - lava_callback - DEBUG - Processing node 69bb4b19a1e2bfb1b3346d71: baseline-arm-clabbe 2026-03-21 01:02:31,643 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb4b19a1e2bfb1b3346d71 2026-03-21 01:02:31,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:02:31,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4b19a1e2bfb1b3346d71 HTTP/1.1" 200 None 2026-03-21 01:02:31,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:02:31,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89da7fbe29c53ae85cce2 HTTP/1.1" 200 None 2026-03-21 01:02:31,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:02:31,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb4b19a1e2bfb1b3346d71 HTTP/1.1" 200 None 2026-03-21 01:02:31,716 - lava_callback - ERROR - Error processing callback for node 69bb4b19a1e2bfb1b3346d71: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59772 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:02:57,804 - lava_callback - INFO - Received callback for node 69bb4b38a1e2bfb1b3346d7d INFO: 172.21.0.1:59780 - "POST /node/69bb4b38a1e2bfb1b3346d7d HTTP/1.0" 202 Accepted 2026-03-21 01:02:57,814 - lava_callback - INFO - Starting async job submit for node 69bb4b38a1e2bfb1b3346d7d 2026-03-21 01:02:57,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:02:57,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4b38a1e2bfb1b3346d7d HTTP/1.1" 200 None 2026-03-21 01:02:57,839 - lava_callback - DEBUG - Processing node 69bb4b38a1e2bfb1b3346d7d: baseline-arm-baylibre 2026-03-21 01:02:57,839 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb4b38a1e2bfb1b3346d7d 2026-03-21 01:02:57,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:02:57,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4b38a1e2bfb1b3346d7d HTTP/1.1" 200 None 2026-03-21 01:02:57,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:02:57,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-21 01:02:57,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:02:57,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb4b38a1e2bfb1b3346d7d HTTP/1.1" 200 None 2026-03-21 01:02:57,909 - lava_callback - ERROR - Error processing callback for node 69bb4b38a1e2bfb1b3346d7d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58538 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51720 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:03:37,770 - lava_callback - INFO - Received callback for node 69bb4b5ca1e2bfb1b3346d8f INFO: 172.21.0.1:51722 - "POST /node/69bb4b5ca1e2bfb1b3346d8f HTTP/1.0" 202 Accepted 2026-03-21 01:03:37,779 - lava_callback - INFO - Starting async job submit for node 69bb4b5ca1e2bfb1b3346d8f 2026-03-21 01:03:37,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:03:37,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4b5ca1e2bfb1b3346d8f HTTP/1.1" 200 None 2026-03-21 01:03:37,806 - lava_callback - DEBUG - Processing node 69bb4b5ca1e2bfb1b3346d8f: baseline-arm-baylibre 2026-03-21 01:03:37,806 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb4b5ca1e2bfb1b3346d8f 2026-03-21 01:03:37,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:03:37,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4b5ca1e2bfb1b3346d8f HTTP/1.1" 200 None 2026-03-21 01:03:37,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:03:37,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-21 01:03:37,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:03:37,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb4b5ca1e2bfb1b3346d8f HTTP/1.1" 200 None 2026-03-21 01:03:37,892 - lava_callback - ERROR - Error processing callback for node 69bb4b5ca1e2bfb1b3346d8f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43940 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:03:59,385 - lava_callback - INFO - Received callback for node 69bde8e5d54b5122738c64c4 INFO: 172.21.0.1:43946 - "POST /node/69bde8e5d54b5122738c64c4 HTTP/1.0" 202 Accepted 2026-03-21 01:03:59,391 - lava_callback - INFO - Starting async job submit for node 69bde8e5d54b5122738c64c4 2026-03-21 01:03:59,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:03:59,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde8e5d54b5122738c64c4 HTTP/1.1" 200 None 2026-03-21 01:03:59,470 - lava_callback - DEBUG - Processing node 69bde8e5d54b5122738c64c4: ltp-ipc 2026-03-21 01:03:59,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:04:00,492 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:04:00,495 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-ipc-69bde8e5d54b5122738c64c4/log.txt.gz 2026-03-21 01:04:00,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:04:01,336 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:04:01,339 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-ipc-69bde8e5d54b5122738c64c4/lava-logs.yaml 2026-03-21 01:04:01,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:04:01,503 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:04:01,504 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-ipc-69bde8e5d54b5122738c64c4/lava_callback.json.gz 2026-03-21 01:04:01,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:04:01,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde8e5d54b5122738c64c4 HTTP/1.1" 200 None 2026-03-21 01:04:01,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:04:01,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-21 01:04:01,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:04:01,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bde8e5d54b5122738c64c4 HTTP/1.1" 200 None 2026-03-21 01:04:01,601 - lava_callback - INFO - Completed processing callback for node 69bde8e5d54b5122738c64c4 INFO: 172.21.0.1:59684 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:04:05,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:04:05,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:59694 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:04:11,722 - lava_callback - INFO - Received callback for node 69bb4b83a1e2bfb1b3346da0 INFO: 172.21.0.1:42256 - "POST /node/69bb4b83a1e2bfb1b3346da0 HTTP/1.0" 202 Accepted 2026-03-21 01:04:11,727 - lava_callback - INFO - Starting async job submit for node 69bb4b83a1e2bfb1b3346da0 2026-03-21 01:04:11,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:04:11,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4b83a1e2bfb1b3346da0 HTTP/1.1" 200 None 2026-03-21 01:04:11,744 - lava_callback - DEBUG - Processing node 69bb4b83a1e2bfb1b3346da0: baseline-arm-clabbe 2026-03-21 01:04:11,744 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb4b83a1e2bfb1b3346da0 2026-03-21 01:04:11,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:04:11,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4b83a1e2bfb1b3346da0 HTTP/1.1" 200 None 2026-03-21 01:04:11,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:04:11,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89da7fbe29c53ae85cce2 HTTP/1.1" 200 None 2026-03-21 01:04:11,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:04:11,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb4b83a1e2bfb1b3346da0 HTTP/1.1" 200 None 2026-03-21 01:04:11,800 - lava_callback - ERROR - Error processing callback for node 69bb4b83a1e2bfb1b3346da0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42264 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38864 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:05:01,447 - lava_callback - INFO - Received callback for node 69bde8e2d54b5122738c64bb INFO: 172.21.0.1:38992 - "POST /node/69bde8e2d54b5122738c64bb HTTP/1.0" 202 Accepted 2026-03-21 01:05:01,457 - lava_callback - INFO - Starting async job submit for node 69bde8e2d54b5122738c64bb 2026-03-21 01:05:01,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:05:01,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde8e2d54b5122738c64bb HTTP/1.1" 200 None 2026-03-21 01:05:01,559 - lava_callback - DEBUG - Processing node 69bde8e2d54b5122738c64bb: ltp-fsx 2026-03-21 01:05:01,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:05:02,129 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:05:02,131 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fsx-69bde8e2d54b5122738c64bb/log.txt.gz 2026-03-21 01:05:02,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:05:02,593 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:05:02,595 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fsx-69bde8e2d54b5122738c64bb/lava-logs.yaml 2026-03-21 01:05:02,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:05:02,758 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:05:02,759 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fsx-69bde8e2d54b5122738c64bb/lava_callback.json.gz 2026-03-21 01:05:02,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:05:02,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde8e2d54b5122738c64bb HTTP/1.1" 200 None 2026-03-21 01:05:02,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:05:02,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-21 01:05:02,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:05:02,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bde8e2d54b5122738c64bb HTTP/1.1" 200 None 2026-03-21 01:05:02,861 - lava_callback - INFO - Completed processing callback for node 69bde8e2d54b5122738c64bb INFO: 172.21.0.1:38998 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:05:05,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:05:05,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:39004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40848 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40856 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:05:19,494 - lava_callback - INFO - Received callback for node 69bde8e7d54b5122738c64c7 INFO: 172.21.0.1:40868 - "POST /node/69bde8e7d54b5122738c64c7 HTTP/1.0" 202 Accepted 2026-03-21 01:05:19,500 - lava_callback - INFO - Starting async job submit for node 69bde8e7d54b5122738c64c7 2026-03-21 01:05:19,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:05:19,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde8e7d54b5122738c64c7 HTTP/1.1" 200 None 2026-03-21 01:05:19,548 - lava_callback - DEBUG - Processing node 69bde8e7d54b5122738c64c7: ltp-pty 2026-03-21 01:05:19,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:05:20,068 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:05:20,070 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-pty-69bde8e7d54b5122738c64c7/log.txt.gz 2026-03-21 01:05:20,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:05:20,521 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:05:20,523 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-pty-69bde8e7d54b5122738c64c7/lava-logs.yaml 2026-03-21 01:05:20,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:05:20,707 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:05:20,709 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-pty-69bde8e7d54b5122738c64c7/lava_callback.json.gz 2026-03-21 01:05:20,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:05:20,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde8e7d54b5122738c64c7 HTTP/1.1" 200 None 2026-03-21 01:05:20,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:05:20,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-21 01:05:20,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:05:20,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bde8e7d54b5122738c64c7 HTTP/1.1" 200 None 2026-03-21 01:05:20,807 - lava_callback - INFO - Completed processing callback for node 69bde8e7d54b5122738c64c7 INFO: 172.21.0.1:42224 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:05:29,322 - lava_callback - INFO - Received callback for node 69bde8e3d54b5122738c64be INFO: 172.21.0.1:42228 - "POST /node/69bde8e3d54b5122738c64be HTTP/1.0" 202 Accepted 2026-03-21 01:05:29,329 - lava_callback - INFO - Starting async job submit for node 69bde8e3d54b5122738c64be 2026-03-21 01:05:29,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:05:29,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde8e3d54b5122738c64be HTTP/1.1" 200 None 2026-03-21 01:05:29,383 - lava_callback - DEBUG - Processing node 69bde8e3d54b5122738c64be: ltp-input 2026-03-21 01:05:29,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:05:29,879 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:05:29,881 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-input-69bde8e3d54b5122738c64be/log.txt.gz 2026-03-21 01:05:30,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:05:30,320 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:05:30,322 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-input-69bde8e3d54b5122738c64be/lava-logs.yaml 2026-03-21 01:05:30,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:05:30,507 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:05:30,509 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-input-69bde8e3d54b5122738c64be/lava_callback.json.gz 2026-03-21 01:05:30,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:05:30,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde8e3d54b5122738c64be HTTP/1.1" 200 None 2026-03-21 01:05:30,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:05:30,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-21 01:05:30,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:05:30,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bde8e3d54b5122738c64be HTTP/1.1" 200 None 2026-03-21 01:05:30,632 - lava_callback - INFO - Completed processing callback for node 69bde8e3d54b5122738c64be 2026-03-21 01:05:35,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:05:35,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:42010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38748 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:05:57,958 - lava_callback - INFO - Received callback for node 69bb4beba1e2bfb1b3346dc4 INFO: 172.21.0.1:38760 - "POST /node/69bb4beba1e2bfb1b3346dc4 HTTP/1.0" 202 Accepted 2026-03-21 01:05:57,964 - lava_callback - INFO - Starting async job submit for node 69bb4beba1e2bfb1b3346dc4 2026-03-21 01:05:57,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:05:57,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4beba1e2bfb1b3346dc4 HTTP/1.1" 200 None 2026-03-21 01:05:57,979 - lava_callback - DEBUG - Processing node 69bb4beba1e2bfb1b3346dc4: baseline-arm-baylibre 2026-03-21 01:05:57,979 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb4beba1e2bfb1b3346dc4 2026-03-21 01:05:57,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:05:57,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4beba1e2bfb1b3346dc4 HTTP/1.1" 200 None 2026-03-21 01:05:57,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:05:58,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-21 01:05:58,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:05:58,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb4beba1e2bfb1b3346dc4 HTTP/1.1" 200 None 2026-03-21 01:05:58,029 - lava_callback - ERROR - Error processing callback for node 69bb4beba1e2bfb1b3346dc4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58598 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44042 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:06:31,806 - lava_callback - INFO - Received callback for node 69bb4c13a1e2bfb1b3346e0f INFO: 172.21.0.1:39014 - "POST /node/69bb4c13a1e2bfb1b3346e0f HTTP/1.0" 202 Accepted 2026-03-21 01:06:31,811 - lava_callback - INFO - Starting async job submit for node 69bb4c13a1e2bfb1b3346e0f 2026-03-21 01:06:31,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:06:31,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4c13a1e2bfb1b3346e0f HTTP/1.1" 200 None 2026-03-21 01:06:31,828 - lava_callback - DEBUG - Processing node 69bb4c13a1e2bfb1b3346e0f: baseline-arm-clabbe 2026-03-21 01:06:31,828 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb4c13a1e2bfb1b3346e0f 2026-03-21 01:06:31,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:06:31,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4c13a1e2bfb1b3346e0f HTTP/1.1" 200 None 2026-03-21 01:06:31,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:06:31,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-21 01:06:31,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:06:31,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb4c13a1e2bfb1b3346e0f HTTP/1.1" 200 None 2026-03-21 01:06:31,895 - lava_callback - ERROR - Error processing callback for node 69bb4c13a1e2bfb1b3346e0f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53588 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:06:57,933 - lava_callback - INFO - Received callback for node 69bb4c2aa1e2bfb1b3346e60 INFO: 172.21.0.1:53594 - "POST /node/69bb4c2aa1e2bfb1b3346e60 HTTP/1.0" 202 Accepted 2026-03-21 01:06:57,938 - lava_callback - INFO - Starting async job submit for node 69bb4c2aa1e2bfb1b3346e60 2026-03-21 01:06:57,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:06:57,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4c2aa1e2bfb1b3346e60 HTTP/1.1" 200 None 2026-03-21 01:06:57,957 - lava_callback - DEBUG - Processing node 69bb4c2aa1e2bfb1b3346e60: baseline-arm64-baylibre 2026-03-21 01:06:57,958 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb4c2aa1e2bfb1b3346e60 2026-03-21 01:06:57,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:06:57,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4c2aa1e2bfb1b3346e60 HTTP/1.1" 200 None 2026-03-21 01:06:57,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:06:57,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-21 01:06:57,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:06:58,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb4c2aa1e2bfb1b3346e60 HTTP/1.1" 200 None 2026-03-21 01:06:58,038 - lava_callback - ERROR - Error processing callback for node 69bb4c2aa1e2bfb1b3346e60: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-21 01:06:58,635 - lava_callback - INFO - Received callback for node 69bb4c2ca1e2bfb1b3346e63 INFO: 172.21.0.1:53604 - "POST /node/69bb4c2ca1e2bfb1b3346e63 HTTP/1.0" 202 Accepted 2026-03-21 01:06:58,641 - lava_callback - INFO - Starting async job submit for node 69bb4c2ca1e2bfb1b3346e63 2026-03-21 01:06:58,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:06:58,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4c2ca1e2bfb1b3346e63 HTTP/1.1" 200 None 2026-03-21 01:06:58,666 - lava_callback - DEBUG - Processing node 69bb4c2ca1e2bfb1b3346e63: baseline-arm64-baylibre 2026-03-21 01:06:58,666 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb4c2ca1e2bfb1b3346e63 2026-03-21 01:06:58,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:06:58,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4c2ca1e2bfb1b3346e63 HTTP/1.1" 200 None 2026-03-21 01:06:58,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:06:58,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-21 01:06:58,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:06:58,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb4c2ca1e2bfb1b3346e63 HTTP/1.1" 200 None 2026-03-21 01:06:58,762 - lava_callback - ERROR - Error processing callback for node 69bb4c2ca1e2bfb1b3346e63: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-21 01:06:59,311 - lava_callback - INFO - Received callback for node 69bb4c28a1e2bfb1b3346e5d INFO: 172.21.0.1:53616 - "POST /node/69bb4c28a1e2bfb1b3346e5d HTTP/1.0" 202 Accepted 2026-03-21 01:06:59,317 - lava_callback - INFO - Starting async job submit for node 69bb4c28a1e2bfb1b3346e5d 2026-03-21 01:06:59,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:06:59,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4c28a1e2bfb1b3346e5d HTTP/1.1" 200 None 2026-03-21 01:06:59,337 - lava_callback - DEBUG - Processing node 69bb4c28a1e2bfb1b3346e5d: baseline-arm64-baylibre 2026-03-21 01:06:59,337 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb4c28a1e2bfb1b3346e5d 2026-03-21 01:06:59,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:06:59,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4c28a1e2bfb1b3346e5d HTTP/1.1" 200 None 2026-03-21 01:06:59,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:06:59,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-21 01:06:59,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:06:59,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb4c28a1e2bfb1b3346e5d HTTP/1.1" 200 None 2026-03-21 01:06:59,415 - lava_callback - ERROR - Error processing callback for node 69bb4c28a1e2bfb1b3346e5d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-21 01:07:00,001 - lava_callback - INFO - Received callback for node 69bb4c2fa1e2bfb1b3346e6a INFO: 172.21.0.1:52710 - "POST /node/69bb4c2fa1e2bfb1b3346e6a HTTP/1.0" 202 Accepted 2026-03-21 01:07:00,007 - lava_callback - INFO - Starting async job submit for node 69bb4c2fa1e2bfb1b3346e6a 2026-03-21 01:07:00,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:07:00,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4c2fa1e2bfb1b3346e6a HTTP/1.1" 200 None 2026-03-21 01:07:00,024 - lava_callback - DEBUG - Processing node 69bb4c2fa1e2bfb1b3346e6a: baseline-arm64-baylibre 2026-03-21 01:07:00,025 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb4c2fa1e2bfb1b3346e6a 2026-03-21 01:07:00,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:07:00,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4c2fa1e2bfb1b3346e6a HTTP/1.1" 200 None 2026-03-21 01:07:00,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:07:00,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-21 01:07:00,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:07:00,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb4c2fa1e2bfb1b3346e6a HTTP/1.1" 200 None 2026-03-21 01:07:00,080 - lava_callback - ERROR - Error processing callback for node 69bb4c2fa1e2bfb1b3346e6a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48160 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48168 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:07:17,969 - lava_callback - INFO - Received callback for node 69bb4c33a1e2bfb1b3346e74 INFO: 172.21.0.1:48170 - "POST /node/69bb4c33a1e2bfb1b3346e74 HTTP/1.0" 202 Accepted 2026-03-21 01:07:17,975 - lava_callback - INFO - Starting async job submit for node 69bb4c33a1e2bfb1b3346e74 2026-03-21 01:07:17,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:07:17,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4c33a1e2bfb1b3346e74 HTTP/1.1" 200 None 2026-03-21 01:07:17,991 - lava_callback - DEBUG - Processing node 69bb4c33a1e2bfb1b3346e74: baseline-arm64-baylibre 2026-03-21 01:07:17,991 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb4c33a1e2bfb1b3346e74 2026-03-21 01:07:17,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:07:18,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4c33a1e2bfb1b3346e74 HTTP/1.1" 200 None 2026-03-21 01:07:18,030 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:07:18,044 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-21 01:07:18,045 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:07:18,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb4c33a1e2bfb1b3346e74 HTTP/1.1" 200 None 2026-03-21 01:07:18,072 - lava_callback - ERROR - Error processing callback for node 69bb4c33a1e2bfb1b3346e74: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-21 01:07:18,630 - lava_callback - INFO - Received callback for node 69bb4c33a1e2bfb1b3346e77 INFO: 172.21.0.1:48174 - "POST /node/69bb4c33a1e2bfb1b3346e77 HTTP/1.0" 202 Accepted 2026-03-21 01:07:18,635 - lava_callback - INFO - Starting async job submit for node 69bb4c33a1e2bfb1b3346e77 2026-03-21 01:07:18,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:07:18,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4c33a1e2bfb1b3346e77 HTTP/1.1" 200 None 2026-03-21 01:07:18,651 - lava_callback - DEBUG - Processing node 69bb4c33a1e2bfb1b3346e77: baseline-arm-baylibre 2026-03-21 01:07:18,652 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb4c33a1e2bfb1b3346e77 2026-03-21 01:07:18,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:07:18,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4c33a1e2bfb1b3346e77 HTTP/1.1" 200 None 2026-03-21 01:07:18,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:07:18,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-21 01:07:18,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:07:18,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb4c33a1e2bfb1b3346e77 HTTP/1.1" 200 None 2026-03-21 01:07:18,719 - lava_callback - ERROR - Error processing callback for node 69bb4c33a1e2bfb1b3346e77: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-21 01:07:19,289 - lava_callback - INFO - Received callback for node 69bb4c31a1e2bfb1b3346e71 INFO: 172.21.0.1:48178 - "POST /node/69bb4c31a1e2bfb1b3346e71 HTTP/1.0" 202 Accepted 2026-03-21 01:07:19,295 - lava_callback - INFO - Starting async job submit for node 69bb4c31a1e2bfb1b3346e71 2026-03-21 01:07:19,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:07:19,310 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4c31a1e2bfb1b3346e71 HTTP/1.1" 200 None 2026-03-21 01:07:19,310 - lava_callback - DEBUG - Processing node 69bb4c31a1e2bfb1b3346e71: baseline-arm64-baylibre 2026-03-21 01:07:19,311 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb4c31a1e2bfb1b3346e71 2026-03-21 01:07:19,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:07:19,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4c31a1e2bfb1b3346e71 HTTP/1.1" 200 None 2026-03-21 01:07:19,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:07:19,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-21 01:07:19,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:07:19,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb4c31a1e2bfb1b3346e71 HTTP/1.1" 200 None 2026-03-21 01:07:19,364 - lava_callback - ERROR - Error processing callback for node 69bb4c31a1e2bfb1b3346e71: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38782 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:07:31,822 - lava_callback - INFO - Received callback for node 69bb4c46a1e2bfb1b3346eb6 INFO: 172.21.0.1:48606 - "POST /node/69bb4c46a1e2bfb1b3346eb6 HTTP/1.0" 202 Accepted 2026-03-21 01:07:31,827 - lava_callback - INFO - Starting async job submit for node 69bb4c46a1e2bfb1b3346eb6 2026-03-21 01:07:31,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:07:31,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4c46a1e2bfb1b3346eb6 HTTP/1.1" 200 None 2026-03-21 01:07:31,843 - lava_callback - DEBUG - Processing node 69bb4c46a1e2bfb1b3346eb6: baseline-arm64-clabbe 2026-03-21 01:07:31,843 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb4c46a1e2bfb1b3346eb6 2026-03-21 01:07:31,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:07:31,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4c46a1e2bfb1b3346eb6 HTTP/1.1" 200 None 2026-03-21 01:07:31,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:07:31,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-21 01:07:31,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:07:31,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb4c46a1e2bfb1b3346eb6 HTTP/1.1" 200 None 2026-03-21 01:07:31,905 - lava_callback - ERROR - Error processing callback for node 69bb4c46a1e2bfb1b3346eb6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-21 01:07:32,491 - lava_callback - INFO - Received callback for node 69bb4c48a1e2bfb1b3346ebd INFO: 172.21.0.1:48614 - "POST /node/69bb4c48a1e2bfb1b3346ebd HTTP/1.0" 202 Accepted 2026-03-21 01:07:32,497 - lava_callback - INFO - Starting async job submit for node 69bb4c48a1e2bfb1b3346ebd 2026-03-21 01:07:32,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:07:32,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4c48a1e2bfb1b3346ebd HTTP/1.1" 200 None 2026-03-21 01:07:32,513 - lava_callback - DEBUG - Processing node 69bb4c48a1e2bfb1b3346ebd: baseline-arm64-clabbe 2026-03-21 01:07:32,513 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb4c48a1e2bfb1b3346ebd 2026-03-21 01:07:32,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:07:32,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4c48a1e2bfb1b3346ebd HTTP/1.1" 200 None 2026-03-21 01:07:32,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:07:32,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-21 01:07:32,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:07:32,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb4c48a1e2bfb1b3346ebd HTTP/1.1" 200 None 2026-03-21 01:07:32,569 - lava_callback - ERROR - Error processing callback for node 69bb4c48a1e2bfb1b3346ebd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-21 01:07:33,154 - lava_callback - INFO - Received callback for node 69bb4c49a1e2bfb1b3346ec0 INFO: 172.21.0.1:48624 - "POST /node/69bb4c49a1e2bfb1b3346ec0 HTTP/1.0" 202 Accepted 2026-03-21 01:07:33,159 - lava_callback - INFO - Starting async job submit for node 69bb4c49a1e2bfb1b3346ec0 2026-03-21 01:07:33,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:07:33,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4c49a1e2bfb1b3346ec0 HTTP/1.1" 200 None 2026-03-21 01:07:33,177 - lava_callback - DEBUG - Processing node 69bb4c49a1e2bfb1b3346ec0: baseline-arm64-clabbe 2026-03-21 01:07:33,177 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb4c49a1e2bfb1b3346ec0 2026-03-21 01:07:33,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:07:33,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4c49a1e2bfb1b3346ec0 HTTP/1.1" 200 None 2026-03-21 01:07:33,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:07:33,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-21 01:07:33,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:07:33,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb4c49a1e2bfb1b3346ec0 HTTP/1.1" 200 None 2026-03-21 01:07:33,262 - lava_callback - ERROR - Error processing callback for node 69bb4c49a1e2bfb1b3346ec0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-21 01:07:33,819 - lava_callback - INFO - Received callback for node 69bb4c4aa1e2bfb1b3346ec3 INFO: 172.21.0.1:48636 - "POST /node/69bb4c4aa1e2bfb1b3346ec3 HTTP/1.0" 202 Accepted 2026-03-21 01:07:33,825 - lava_callback - INFO - Starting async job submit for node 69bb4c4aa1e2bfb1b3346ec3 2026-03-21 01:07:33,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:07:33,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4c4aa1e2bfb1b3346ec3 HTTP/1.1" 200 None 2026-03-21 01:07:33,841 - lava_callback - DEBUG - Processing node 69bb4c4aa1e2bfb1b3346ec3: baseline-arm64-clabbe 2026-03-21 01:07:33,841 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb4c4aa1e2bfb1b3346ec3 2026-03-21 01:07:33,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:07:33,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4c4aa1e2bfb1b3346ec3 HTTP/1.1" 200 None 2026-03-21 01:07:33,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:07:33,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-21 01:07:33,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:07:33,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb4c4aa1e2bfb1b3346ec3 HTTP/1.1" 200 None 2026-03-21 01:07:33,894 - lava_callback - ERROR - Error processing callback for node 69bb4c4aa1e2bfb1b3346ec3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-21 01:07:34,508 - lava_callback - INFO - Received callback for node 69bb4c4ca1e2bfb1b3346ed3 INFO: 172.21.0.1:48644 - "POST /node/69bb4c4ca1e2bfb1b3346ed3 HTTP/1.0" 202 Accepted 2026-03-21 01:07:34,513 - lava_callback - INFO - Starting async job submit for node 69bb4c4ca1e2bfb1b3346ed3 2026-03-21 01:07:34,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:07:34,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4c4ca1e2bfb1b3346ed3 HTTP/1.1" 200 None 2026-03-21 01:07:34,537 - lava_callback - DEBUG - Processing node 69bb4c4ca1e2bfb1b3346ed3: baseline-arm64-clabbe 2026-03-21 01:07:34,538 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb4c4ca1e2bfb1b3346ed3 2026-03-21 01:07:34,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:07:34,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4c4ca1e2bfb1b3346ed3 HTTP/1.1" 200 None 2026-03-21 01:07:34,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:07:34,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-21 01:07:34,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:07:34,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb4c4ca1e2bfb1b3346ed3 HTTP/1.1" 200 None 2026-03-21 01:07:34,591 - lava_callback - ERROR - Error processing callback for node 69bb4c4ca1e2bfb1b3346ed3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-21 01:07:35,166 - lava_callback - INFO - Received callback for node 69bb4c4da1e2bfb1b3346ed6 INFO: 172.21.0.1:48646 - "POST /node/69bb4c4da1e2bfb1b3346ed6 HTTP/1.0" 202 Accepted 2026-03-21 01:07:35,172 - lava_callback - INFO - Starting async job submit for node 69bb4c4da1e2bfb1b3346ed6 2026-03-21 01:07:35,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:07:35,187 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4c4da1e2bfb1b3346ed6 HTTP/1.1" 200 None 2026-03-21 01:07:35,188 - lava_callback - DEBUG - Processing node 69bb4c4da1e2bfb1b3346ed6: baseline-arm64-clabbe 2026-03-21 01:07:35,188 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb4c4da1e2bfb1b3346ed6 2026-03-21 01:07:35,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:07:35,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4c4da1e2bfb1b3346ed6 HTTP/1.1" 200 None 2026-03-21 01:07:35,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:07:35,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-21 01:07:35,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:07:35,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb4c4da1e2bfb1b3346ed6 HTTP/1.1" 200 None 2026-03-21 01:07:35,246 - lava_callback - ERROR - Error processing callback for node 69bb4c4da1e2bfb1b3346ed6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35106 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35112 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:08:18,024 - lava_callback - INFO - Received callback for node 69bb4c7ba1e2bfb1b3346f66 INFO: 172.21.0.1:35114 - "POST /node/69bb4c7ba1e2bfb1b3346f66 HTTP/1.0" 202 Accepted 2026-03-21 01:08:18,030 - lava_callback - INFO - Starting async job submit for node 69bb4c7ba1e2bfb1b3346f66 2026-03-21 01:08:18,030 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:08:18,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4c7ba1e2bfb1b3346f66 HTTP/1.1" 200 None 2026-03-21 01:08:18,045 - lava_callback - DEBUG - Processing node 69bb4c7ba1e2bfb1b3346f66: baseline-arm-baylibre 2026-03-21 01:08:18,046 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb4c7ba1e2bfb1b3346f66 2026-03-21 01:08:18,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:08:18,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4c7ba1e2bfb1b3346f66 HTTP/1.1" 200 None 2026-03-21 01:08:18,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:08:18,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-21 01:08:18,076 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:08:18,110 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb4c7ba1e2bfb1b3346f66 HTTP/1.1" 200 None 2026-03-21 01:08:18,115 - lava_callback - ERROR - Error processing callback for node 69bb4c7ba1e2bfb1b3346f66: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-21 01:08:19,550 - lava_callback - INFO - Received callback for node 69bde8e8d54b5122738c64ca INFO: 172.21.0.1:35126 - "POST /node/69bde8e8d54b5122738c64ca HTTP/1.0" 202 Accepted 2026-03-21 01:08:19,556 - lava_callback - INFO - Starting async job submit for node 69bde8e8d54b5122738c64ca 2026-03-21 01:08:19,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:08:19,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde8e8d54b5122738c64ca HTTP/1.1" 200 None 2026-03-21 01:08:19,605 - lava_callback - DEBUG - Processing node 69bde8e8d54b5122738c64ca: ltp-sched 2026-03-21 01:08:19,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:08:20,143 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:08:20,145 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-sched-69bde8e8d54b5122738c64ca/log.txt.gz 2026-03-21 01:08:20,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:08:20,614 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:08:20,616 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-sched-69bde8e8d54b5122738c64ca/lava-logs.yaml 2026-03-21 01:08:20,625 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:08:20,791 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:08:20,793 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-sched-69bde8e8d54b5122738c64ca/lava_callback.json.gz 2026-03-21 01:08:20,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:08:20,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde8e8d54b5122738c64ca HTTP/1.1" 200 None 2026-03-21 01:08:20,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:08:20,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-21 01:08:20,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:08:20,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bde8e8d54b5122738c64ca HTTP/1.1" 200 None 2026-03-21 01:08:20,889 - lava_callback - INFO - Completed processing callback for node 69bde8e8d54b5122738c64ca INFO: 172.21.0.1:49390 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:08:35,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:08:35,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:56794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40714 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:08:58,047 - lava_callback - INFO - Received callback for node 69bb4ca0a1e2bfb1b3346fb3 INFO: 172.21.0.1:40724 - "POST /node/69bb4ca0a1e2bfb1b3346fb3 HTTP/1.0" 202 Accepted 2026-03-21 01:08:58,053 - lava_callback - INFO - Starting async job submit for node 69bb4ca0a1e2bfb1b3346fb3 2026-03-21 01:08:58,054 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:08:58,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4ca0a1e2bfb1b3346fb3 HTTP/1.1" 200 None 2026-03-21 01:08:58,073 - lava_callback - DEBUG - Processing node 69bb4ca0a1e2bfb1b3346fb3: baseline-arm-baylibre 2026-03-21 01:08:58,073 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb4ca0a1e2bfb1b3346fb3 2026-03-21 01:08:58,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:08:58,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4ca0a1e2bfb1b3346fb3 HTTP/1.1" 200 None 2026-03-21 01:08:58,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:08:58,105 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-21 01:08:58,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:08:58,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb4ca0a1e2bfb1b3346fb3 HTTP/1.1" 200 None 2026-03-21 01:08:58,128 - lava_callback - ERROR - Error processing callback for node 69bb4ca0a1e2bfb1b3346fb3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41458 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41472 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:09:19,524 - lava_callback - INFO - Received callback for node 69bde8ead54b5122738c64cd INFO: 172.21.0.1:41478 - "POST /node/69bde8ead54b5122738c64cd HTTP/1.0" 202 Accepted 2026-03-21 01:09:19,532 - lava_callback - INFO - Starting async job submit for node 69bde8ead54b5122738c64cd 2026-03-21 01:09:19,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:09:19,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde8ead54b5122738c64cd HTTP/1.1" 200 None 2026-03-21 01:09:19,586 - lava_callback - DEBUG - Processing node 69bde8ead54b5122738c64cd: ltp-smoke 2026-03-21 01:09:19,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:09:20,094 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:09:20,095 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69bde8ead54b5122738c64cd/log.txt.gz 2026-03-21 01:09:20,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:09:20,535 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:09:20,537 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69bde8ead54b5122738c64cd/lava-logs.yaml 2026-03-21 01:09:20,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:09:20,725 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:09:20,727 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69bde8ead54b5122738c64cd/lava_callback.json.gz 2026-03-21 01:09:20,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:09:20,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde8ead54b5122738c64cd HTTP/1.1" 200 None 2026-03-21 01:09:20,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:09:20,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-21 01:09:20,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:09:20,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bde8ead54b5122738c64cd HTTP/1.1" 200 None 2026-03-21 01:09:20,819 - lava_callback - INFO - Completed processing callback for node 69bde8ead54b5122738c64cd 2026-03-21 01:09:21,122 - lava_callback - INFO - Received callback for node 69bde8efd54b5122738c64db INFO: 172.21.0.1:55722 - "POST /node/69bde8efd54b5122738c64db HTTP/1.0" 202 Accepted 2026-03-21 01:09:21,129 - lava_callback - INFO - Starting async job submit for node 69bde8efd54b5122738c64db 2026-03-21 01:09:21,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:09:21,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde8efd54b5122738c64db HTTP/1.1" 200 None 2026-03-21 01:09:21,174 - lava_callback - DEBUG - Processing node 69bde8efd54b5122738c64db: ltp-timers 2026-03-21 01:09:21,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:09:21,701 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:09:21,703 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-timers-69bde8efd54b5122738c64db/log.txt.gz 2026-03-21 01:09:21,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:09:22,135 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:09:22,137 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-timers-69bde8efd54b5122738c64db/lava-logs.yaml 2026-03-21 01:09:22,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:09:22,324 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:09:22,325 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-timers-69bde8efd54b5122738c64db/lava_callback.json.gz 2026-03-21 01:09:22,351 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:09:22,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde8efd54b5122738c64db HTTP/1.1" 200 None 2026-03-21 01:09:22,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:09:22,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-21 01:09:22,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:09:22,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bde8efd54b5122738c64db HTTP/1.1" 200 None 2026-03-21 01:09:22,420 - lava_callback - INFO - Completed processing callback for node 69bde8efd54b5122738c64db INFO: 172.21.0.1:55738 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:09:29,349 - lava_callback - INFO - Received callback for node 69bde8edd54b5122738c64d7 INFO: 172.21.0.1:55746 - "POST /node/69bde8edd54b5122738c64d7 HTTP/1.0" 202 Accepted 2026-03-21 01:09:29,356 - lava_callback - INFO - Starting async job submit for node 69bde8edd54b5122738c64d7 2026-03-21 01:09:29,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:09:29,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde8edd54b5122738c64d7 HTTP/1.1" 200 None 2026-03-21 01:09:29,395 - lava_callback - DEBUG - Processing node 69bde8edd54b5122738c64d7: ltp-syscalls-ipc 2026-03-21 01:09:29,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:09:29,892 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:09:29,894 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69bde8edd54b5122738c64d7/log.txt.gz 2026-03-21 01:09:30,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:09:30,296 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:09:30,298 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69bde8edd54b5122738c64d7/lava-logs.yaml 2026-03-21 01:09:30,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:09:30,456 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:09:30,457 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69bde8edd54b5122738c64d7/lava_callback.json.gz 2026-03-21 01:09:30,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:09:30,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde8edd54b5122738c64d7 HTTP/1.1" 200 None 2026-03-21 01:09:30,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:09:30,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-21 01:09:30,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:09:30,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bde8edd54b5122738c64d7 HTTP/1.1" 200 None 2026-03-21 01:09:30,551 - lava_callback - INFO - Completed processing callback for node 69bde8edd54b5122738c64d7 2026-03-21 01:09:35,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:09:35,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:42580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36292 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36306 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:10:18,199 - lava_callback - INFO - Received callback for node 69bb4ce6a1e2bfb1b334701f INFO: 172.21.0.1:36316 - "POST /node/69bb4ce6a1e2bfb1b334701f HTTP/1.0" 202 Accepted 2026-03-21 01:10:18,205 - lava_callback - INFO - Starting async job submit for node 69bb4ce6a1e2bfb1b334701f 2026-03-21 01:10:18,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:10:18,220 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4ce6a1e2bfb1b334701f HTTP/1.1" 200 None 2026-03-21 01:10:18,221 - lava_callback - DEBUG - Processing node 69bb4ce6a1e2bfb1b334701f: baseline-arm-baylibre 2026-03-21 01:10:18,221 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb4ce6a1e2bfb1b334701f 2026-03-21 01:10:18,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:10:18,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4ce6a1e2bfb1b334701f HTTP/1.1" 200 None 2026-03-21 01:10:18,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:10:18,254 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-21 01:10:18,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:10:18,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb4ce6a1e2bfb1b334701f HTTP/1.1" 200 None 2026-03-21 01:10:18,280 - lava_callback - ERROR - Error processing callback for node 69bb4ce6a1e2bfb1b334701f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33464 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.21.0.1:33480 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35752 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:11:32,035 - lava_callback - INFO - Received callback for node 69bb4d2fa1e2bfb1b3347041 INFO: 172.21.0.1:38076 - "POST /node/69bb4d2fa1e2bfb1b3347041 HTTP/1.0" 202 Accepted 2026-03-21 01:11:32,041 - lava_callback - INFO - Starting async job submit for node 69bb4d2fa1e2bfb1b3347041 2026-03-21 01:11:32,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:11:32,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4d2fa1e2bfb1b3347041 HTTP/1.1" 200 None 2026-03-21 01:11:32,058 - lava_callback - DEBUG - Processing node 69bb4d2fa1e2bfb1b3347041: baseline-arm-clabbe 2026-03-21 01:11:32,058 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb4d2fa1e2bfb1b3347041 2026-03-21 01:11:32,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:11:32,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4d2fa1e2bfb1b3347041 HTTP/1.1" 200 None 2026-03-21 01:11:32,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:11:32,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-21 01:11:32,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:11:32,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb4d2fa1e2bfb1b3347041 HTTP/1.1" 200 None 2026-03-21 01:11:32,121 - lava_callback - ERROR - Error processing callback for node 69bb4d2fa1e2bfb1b3347041: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33086 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:11:58,369 - lava_callback - INFO - Received callback for node 69bb4d4fa1e2bfb1b3347053 INFO: 172.21.0.1:33096 - "POST /node/69bb4d4fa1e2bfb1b3347053 HTTP/1.0" 202 Accepted 2026-03-21 01:11:58,379 - lava_callback - INFO - Starting async job submit for node 69bb4d4fa1e2bfb1b3347053 2026-03-21 01:11:58,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:11:58,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4d4fa1e2bfb1b3347053 HTTP/1.1" 200 None 2026-03-21 01:11:58,413 - lava_callback - DEBUG - Processing node 69bb4d4fa1e2bfb1b3347053: baseline-arm-baylibre 2026-03-21 01:11:58,413 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb4d4fa1e2bfb1b3347053 2026-03-21 01:11:58,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:11:58,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4d4fa1e2bfb1b3347053 HTTP/1.1" 200 None 2026-03-21 01:11:58,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:11:58,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-21 01:11:58,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:11:58,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb4d4fa1e2bfb1b3347053 HTTP/1.1" 200 None 2026-03-21 01:11:58,516 - lava_callback - ERROR - Error processing callback for node 69bb4d4fa1e2bfb1b3347053: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56338 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52388 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:12:39,688 - lava_callback - INFO - Received callback for node 69bde8f0d54b5122738c64de INFO: 172.21.0.1:52398 - "POST /node/69bde8f0d54b5122738c64de HTTP/1.0" 202 Accepted 2026-03-21 01:12:39,698 - lava_callback - INFO - Starting async job submit for node 69bde8f0d54b5122738c64de 2026-03-21 01:12:39,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:12:39,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde8f0d54b5122738c64de HTTP/1.1" 200 None 2026-03-21 01:12:39,780 - lava_callback - DEBUG - Processing node 69bde8f0d54b5122738c64de: ltp-watchqueue 2026-03-21 01:12:40,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:12:40,652 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:12:40,655 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69bde8f0d54b5122738c64de/log.txt.gz 2026-03-21 01:12:40,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:12:41,409 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:12:41,412 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69bde8f0d54b5122738c64de/lava-logs.yaml 2026-03-21 01:12:41,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:12:41,590 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:12:41,592 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69bde8f0d54b5122738c64de/lava_callback.json.gz 2026-03-21 01:12:41,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:12:41,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde8f0d54b5122738c64de HTTP/1.1" 200 None 2026-03-21 01:12:41,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:12:41,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-21 01:12:41,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:12:41,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bde8f0d54b5122738c64de HTTP/1.1" 200 None 2026-03-21 01:12:41,683 - lava_callback - INFO - Completed processing callback for node 69bde8f0d54b5122738c64de INFO: 172.21.0.1:43086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48514 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:13:05,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:13:06,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:48528 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:13:12,074 - lava_callback - INFO - Received callback for node 69bb4d99a1e2bfb1b334706d INFO: 172.21.0.1:55092 - "POST /node/69bb4d99a1e2bfb1b334706d HTTP/1.0" 202 Accepted 2026-03-21 01:13:12,080 - lava_callback - INFO - Starting async job submit for node 69bb4d99a1e2bfb1b334706d 2026-03-21 01:13:12,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:13:12,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4d99a1e2bfb1b334706d HTTP/1.1" 200 None 2026-03-21 01:13:12,097 - lava_callback - DEBUG - Processing node 69bb4d99a1e2bfb1b334706d: baseline-arm-clabbe 2026-03-21 01:13:12,098 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb4d99a1e2bfb1b334706d 2026-03-21 01:13:12,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:13:12,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4d99a1e2bfb1b334706d HTTP/1.1" 200 None 2026-03-21 01:13:12,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:13:12,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-21 01:13:12,129 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:13:12,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb4d99a1e2bfb1b334706d HTTP/1.1" 200 None 2026-03-21 01:13:12,175 - lava_callback - ERROR - Error processing callback for node 69bb4d99a1e2bfb1b334706d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55100 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35800 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:13:52,091 - lava_callback - INFO - Received callback for node 69bb4dbfa1e2bfb1b334707c INFO: 172.21.0.1:42280 - "POST /node/69bb4dbfa1e2bfb1b334707c HTTP/1.0" 202 Accepted 2026-03-21 01:13:52,096 - lava_callback - INFO - Starting async job submit for node 69bb4dbfa1e2bfb1b334707c 2026-03-21 01:13:52,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:13:52,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4dbfa1e2bfb1b334707c HTTP/1.1" 200 None 2026-03-21 01:13:52,116 - lava_callback - DEBUG - Processing node 69bb4dbfa1e2bfb1b334707c: baseline-arm-clabbe 2026-03-21 01:13:52,116 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb4dbfa1e2bfb1b334707c 2026-03-21 01:13:52,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:13:52,129 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4dbfa1e2bfb1b334707c HTTP/1.1" 200 None 2026-03-21 01:13:52,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:13:52,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-21 01:13:52,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:13:52,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb4dbfa1e2bfb1b334707c HTTP/1.1" 200 None 2026-03-21 01:13:52,171 - lava_callback - ERROR - Error processing callback for node 69bb4dbfa1e2bfb1b334707c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45728 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45926 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:14:32,140 - lava_callback - INFO - Received callback for node 69bb4de5a1e2bfb1b334708b INFO: 172.21.0.1:49772 - "POST /node/69bb4de5a1e2bfb1b334708b HTTP/1.0" 202 Accepted 2026-03-21 01:14:32,146 - lava_callback - INFO - Starting async job submit for node 69bb4de5a1e2bfb1b334708b 2026-03-21 01:14:32,147 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:14:32,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4de5a1e2bfb1b334708b HTTP/1.1" 200 None 2026-03-21 01:14:32,178 - lava_callback - DEBUG - Processing node 69bb4de5a1e2bfb1b334708b: baseline-arm-clabbe 2026-03-21 01:14:32,178 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb4de5a1e2bfb1b334708b 2026-03-21 01:14:32,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:14:32,193 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4de5a1e2bfb1b334708b HTTP/1.1" 200 None 2026-03-21 01:14:32,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:14:32,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-21 01:14:32,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:14:32,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb4de5a1e2bfb1b334708b HTTP/1.1" 200 None 2026-03-21 01:14:32,237 - lava_callback - ERROR - Error processing callback for node 69bb4de5a1e2bfb1b334708b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39748 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:14:58,449 - lava_callback - INFO - Received callback for node 69bb4e07a1e2bfb1b3347098 INFO: 172.21.0.1:39764 - "POST /node/69bb4e07a1e2bfb1b3347098 HTTP/1.0" 202 Accepted 2026-03-21 01:14:58,455 - lava_callback - INFO - Starting async job submit for node 69bb4e07a1e2bfb1b3347098 2026-03-21 01:14:58,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:14:58,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4e07a1e2bfb1b3347098 HTTP/1.1" 200 None 2026-03-21 01:14:58,469 - lava_callback - DEBUG - Processing node 69bb4e07a1e2bfb1b3347098: baseline-arm-baylibre 2026-03-21 01:14:58,470 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb4e07a1e2bfb1b3347098 2026-03-21 01:14:58,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:14:58,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4e07a1e2bfb1b3347098 HTTP/1.1" 200 None 2026-03-21 01:14:58,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:14:58,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-21 01:14:58,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:14:58,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb4e07a1e2bfb1b3347098 HTTP/1.1" 200 None 2026-03-21 01:14:58,543 - lava_callback - ERROR - Error processing callback for node 69bb4e07a1e2bfb1b3347098: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45088 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60042 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:15:38,430 - lava_callback - INFO - Received callback for node 69bb4e2aa1e2bfb1b33470a8 INFO: 172.21.0.1:60044 - "POST /node/69bb4e2aa1e2bfb1b33470a8 HTTP/1.0" 202 Accepted 2026-03-21 01:15:38,435 - lava_callback - INFO - Starting async job submit for node 69bb4e2aa1e2bfb1b33470a8 2026-03-21 01:15:38,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:15:38,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4e2aa1e2bfb1b33470a8 HTTP/1.1" 200 None 2026-03-21 01:15:38,451 - lava_callback - DEBUG - Processing node 69bb4e2aa1e2bfb1b33470a8: baseline-arm-baylibre 2026-03-21 01:15:38,451 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb4e2aa1e2bfb1b33470a8 2026-03-21 01:15:38,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:15:38,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4e2aa1e2bfb1b33470a8 HTTP/1.1" 200 None 2026-03-21 01:15:38,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:15:38,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-21 01:15:38,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:15:38,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb4e2aa1e2bfb1b33470a8 HTTP/1.1" 200 None 2026-03-21 01:15:38,519 - lava_callback - ERROR - Error processing callback for node 69bb4e2aa1e2bfb1b33470a8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58028 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58038 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:16:18,503 - lava_callback - INFO - Received callback for node 69bb4e52a1e2bfb1b33470bf INFO: 172.21.0.1:58048 - "POST /node/69bb4e52a1e2bfb1b33470bf HTTP/1.0" 202 Accepted 2026-03-21 01:16:18,509 - lava_callback - INFO - Starting async job submit for node 69bb4e52a1e2bfb1b33470bf 2026-03-21 01:16:18,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:16:18,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4e52a1e2bfb1b33470bf HTTP/1.1" 200 None 2026-03-21 01:16:18,525 - lava_callback - DEBUG - Processing node 69bb4e52a1e2bfb1b33470bf: baseline-arm-baylibre 2026-03-21 01:16:18,525 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb4e52a1e2bfb1b33470bf 2026-03-21 01:16:18,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:16:18,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4e52a1e2bfb1b33470bf HTTP/1.1" 200 None 2026-03-21 01:16:18,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:16:18,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-21 01:16:18,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:16:18,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb4e52a1e2bfb1b33470bf HTTP/1.1" 200 None 2026-03-21 01:16:18,583 - lava_callback - ERROR - Error processing callback for node 69bb4e52a1e2bfb1b33470bf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53178 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:16:31,692 - lava_callback - INFO - Received callback for node 69bded47d54b5122738c675e INFO: 172.21.0.1:51016 - "POST /node/69bded47d54b5122738c675e HTTP/1.0" 202 Accepted 2026-03-21 01:16:31,699 - lava_callback - INFO - Starting async job submit for node 69bded47d54b5122738c675e 2026-03-21 01:16:31,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:16:31,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bded47d54b5122738c675e HTTP/1.1" 200 None 2026-03-21 01:16:31,771 - lava_callback - DEBUG - Processing node 69bded47d54b5122738c675e: ltp-smoke 2026-03-21 01:16:32,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:16:32,376 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:16:32,377 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69bded47d54b5122738c675e/log.txt.gz 2026-03-21 01:16:32,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:16:33,027 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:16:33,029 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69bded47d54b5122738c675e/lava-logs.yaml 2026-03-21 01:16:33,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:16:33,195 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:16:33,197 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69bded47d54b5122738c675e/lava_callback.json.gz 2026-03-21 01:16:33,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:16:33,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bded47d54b5122738c675e HTTP/1.1" 200 None 2026-03-21 01:16:33,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:16:33,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde35fd54b5122738c3d99 HTTP/1.1" 200 None 2026-03-21 01:16:33,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:16:33,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bded47d54b5122738c675e HTTP/1.1" 200 None 2026-03-21 01:16:33,516 - lava_callback - INFO - Completed processing callback for node 69bded47d54b5122738c675e 2026-03-21 01:16:36,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:16:36,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:51032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41320 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:16:52,229 - lava_callback - INFO - Received callback for node 69bb4e7aa1e2bfb1b33470d2 INFO: 172.21.0.1:38152 - "POST /node/69bb4e7aa1e2bfb1b33470d2 HTTP/1.0" 202 Accepted 2026-03-21 01:16:52,234 - lava_callback - INFO - Starting async job submit for node 69bb4e7aa1e2bfb1b33470d2 2026-03-21 01:16:52,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:16:52,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4e7aa1e2bfb1b33470d2 HTTP/1.1" 200 None 2026-03-21 01:16:52,250 - lava_callback - DEBUG - Processing node 69bb4e7aa1e2bfb1b33470d2: baseline-arm-clabbe 2026-03-21 01:16:52,251 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb4e7aa1e2bfb1b33470d2 2026-03-21 01:16:52,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:16:52,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4e7aa1e2bfb1b33470d2 HTTP/1.1" 200 None 2026-03-21 01:16:52,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:16:52,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b35934974c91e64560e49d HTTP/1.1" 200 None 2026-03-21 01:16:52,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:16:52,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb4e7aa1e2bfb1b33470d2 HTTP/1.1" 200 None 2026-03-21 01:16:52,304 - lava_callback - ERROR - Error processing callback for node 69bb4e7aa1e2bfb1b33470d2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51892 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51894 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:17:18,534 - lava_callback - INFO - Received callback for node 69bb4e9ba1e2bfb1b33470de INFO: 172.21.0.1:51904 - "POST /node/69bb4e9ba1e2bfb1b33470de HTTP/1.0" 202 Accepted 2026-03-21 01:17:18,539 - lava_callback - INFO - Starting async job submit for node 69bb4e9ba1e2bfb1b33470de 2026-03-21 01:17:18,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:17:18,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4e9ba1e2bfb1b33470de HTTP/1.1" 200 None 2026-03-21 01:17:18,554 - lava_callback - DEBUG - Processing node 69bb4e9ba1e2bfb1b33470de: baseline-arm-baylibre 2026-03-21 01:17:18,555 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb4e9ba1e2bfb1b33470de 2026-03-21 01:17:18,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:17:18,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4e9ba1e2bfb1b33470de HTTP/1.1" 200 None 2026-03-21 01:17:18,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:17:18,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-21 01:17:18,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:17:18,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb4e9ba1e2bfb1b33470de HTTP/1.1" 200 None 2026-03-21 01:17:18,618 - lava_callback - ERROR - Error processing callback for node 69bb4e9ba1e2bfb1b33470de: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49892 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:17:29,516 - lava_callback - INFO - Received callback for node 69bd7ad90e93e2314723e72b INFO: 172.21.0.1:49900 - "POST /node/69bd7ad90e93e2314723e72b HTTP/1.0" 202 Accepted 2026-03-21 01:17:29,522 - lava_callback - INFO - Starting async job submit for node 69bd7ad90e93e2314723e72b 2026-03-21 01:17:29,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:17:29,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd7ad90e93e2314723e72b HTTP/1.1" 200 None 2026-03-21 01:17:29,558 - lava_callback - DEBUG - Processing node 69bd7ad90e93e2314723e72b: baseline-arm64-broonie 2026-03-21 01:17:29,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:17:29,907 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:17:29,909 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69bd7ad90e93e2314723e72b/log.txt.gz 2026-03-21 01:17:30,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:17:30,246 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:17:30,248 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69bd7ad90e93e2314723e72b/lava-logs.yaml 2026-03-21 01:17:30,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:17:30,417 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:17:30,419 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69bd7ad90e93e2314723e72b/lava_callback.json.gz 2026-03-21 01:17:30,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:17:30,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd7ad90e93e2314723e72b HTTP/1.1" 200 None 2026-03-21 01:17:30,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:17:30,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-21 01:17:30,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:17:30,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd7ad90e93e2314723e72b HTTP/1.1" 200 None 2026-03-21 01:17:30,509 - lava_callback - INFO - Completed processing callback for node 69bd7ad90e93e2314723e72b 2026-03-21 01:17:36,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:17:36,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:38464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50970 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:18:10,560 - lava_callback - INFO - Received callback for node 69bdedded54b5122738c67a6 INFO: 172.21.0.1:35526 - "POST /node/69bdedded54b5122738c67a6 HTTP/1.0" 202 Accepted 2026-03-21 01:18:10,567 - lava_callback - INFO - Starting async job submit for node 69bdedded54b5122738c67a6 2026-03-21 01:18:10,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:18:10,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdedded54b5122738c67a6 HTTP/1.1" 200 None 2026-03-21 01:18:10,634 - lava_callback - DEBUG - Processing node 69bdedded54b5122738c67a6: baseline-arm 2026-03-21 01:18:11,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:18:11,371 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:18:11,373 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69bdedded54b5122738c67a6/log.txt.gz 2026-03-21 01:18:11,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:18:11,969 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:18:11,971 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69bdedded54b5122738c67a6/lava-logs.yaml 2026-03-21 01:18:11,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:18:12,148 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:18:12,150 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69bdedded54b5122738c67a6/lava_callback.json.gz DEBUG: Setup failed for 69bdedded54b5122738c67a6. Transitting job node result: incomplete -> fail 2026-03-21 01:18:12,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:18:12,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdedded54b5122738c67a6 HTTP/1.1" 200 None 2026-03-21 01:18:12,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:18:12,290 - lava_callback - INFO - Received callback for node 69bb4ec5a1e2bfb1b33470ee INFO: 172.21.0.1:35530 - "POST /node/69bb4ec5a1e2bfb1b33470ee HTTP/1.0" 202 Accepted 2026-03-21 01:18:12,295 - lava_callback - INFO - Starting async job submit for node 69bb4ec5a1e2bfb1b33470ee 2026-03-21 01:18:12,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:18:12,302 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-21 01:18:12,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:18:12,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4ec5a1e2bfb1b33470ee HTTP/1.1" 200 None 2026-03-21 01:18:12,312 - lava_callback - DEBUG - Processing node 69bb4ec5a1e2bfb1b33470ee: baseline-arm-clabbe 2026-03-21 01:18:12,313 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb4ec5a1e2bfb1b33470ee 2026-03-21 01:18:12,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:18:12,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdedded54b5122738c67a6 HTTP/1.1" 200 None 2026-03-21 01:18:12,374 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4ec5a1e2bfb1b33470ee HTTP/1.1" 200 None 2026-03-21 01:18:12,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:18:12,479 - lava_callback - INFO - Completed processing callback for node 69bdedded54b5122738c67a6 2026-03-21 01:18:12,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-21 01:18:12,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:18:12,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb4ec5a1e2bfb1b33470ee HTTP/1.1" 200 None 2026-03-21 01:18:12,525 - lava_callback - ERROR - Error processing callback for node 69bb4ec5a1e2bfb1b33470ee: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35542 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43112 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:18:29,551 - lava_callback - INFO - Received callback for node 69bdd5720e93e23147242cd4 INFO: 172.21.0.1:43114 - "POST /node/69bdd5720e93e23147242cd4 HTTP/1.0" 202 Accepted 2026-03-21 01:18:29,558 - lava_callback - INFO - Starting async job submit for node 69bdd5720e93e23147242cd4 2026-03-21 01:18:29,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:18:29,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd5720e93e23147242cd4 HTTP/1.1" 200 None 2026-03-21 01:18:29,604 - lava_callback - DEBUG - Processing node 69bdd5720e93e23147242cd4: baseline-arm64-broonie 2026-03-21 01:18:29,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:18:29,955 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:18:29,956 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69bdd5720e93e23147242cd4/log.txt.gz 2026-03-21 01:18:30,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:18:30,287 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:18:30,289 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69bdd5720e93e23147242cd4/lava-logs.yaml 2026-03-21 01:18:30,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:18:30,428 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:18:30,430 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69bdd5720e93e23147242cd4/lava_callback.json.gz 2026-03-21 01:18:30,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:18:30,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd5720e93e23147242cd4 HTTP/1.1" 200 None 2026-03-21 01:18:30,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:18:30,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-21 01:18:30,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:18:30,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdd5720e93e23147242cd4 HTTP/1.1" 200 None 2026-03-21 01:18:30,544 - lava_callback - INFO - Completed processing callback for node 69bdd5720e93e23147242cd4 2026-03-21 01:18:36,045 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:18:36,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:34244 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:18:38,640 - lava_callback - INFO - Received callback for node 69bb4ee8a1e2bfb1b33470fa INFO: 172.21.0.1:34252 - "POST /node/69bb4ee8a1e2bfb1b33470fa HTTP/1.0" 202 Accepted 2026-03-21 01:18:38,645 - lava_callback - INFO - Starting async job submit for node 69bb4ee8a1e2bfb1b33470fa 2026-03-21 01:18:38,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:18:38,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4ee8a1e2bfb1b33470fa HTTP/1.1" 200 None 2026-03-21 01:18:38,673 - lava_callback - DEBUG - Processing node 69bb4ee8a1e2bfb1b33470fa: baseline-arm-baylibre 2026-03-21 01:18:38,673 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb4ee8a1e2bfb1b33470fa 2026-03-21 01:18:38,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:18:38,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4ee8a1e2bfb1b33470fa HTTP/1.1" 200 None 2026-03-21 01:18:38,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:18:38,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-21 01:18:38,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:18:38,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb4ee8a1e2bfb1b33470fa HTTP/1.1" 200 None 2026-03-21 01:18:38,728 - lava_callback - ERROR - Error processing callback for node 69bb4ee8a1e2bfb1b33470fa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34832 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34834 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:19:18,646 - lava_callback - INFO - Received callback for node 69bb4f0ca1e2bfb1b3347106 INFO: 172.21.0.1:34844 - "POST /node/69bb4f0ca1e2bfb1b3347106 HTTP/1.0" 202 Accepted 2026-03-21 01:19:18,652 - lava_callback - INFO - Starting async job submit for node 69bb4f0ca1e2bfb1b3347106 2026-03-21 01:19:18,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:19:18,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4f0ca1e2bfb1b3347106 HTTP/1.1" 200 None 2026-03-21 01:19:18,668 - lava_callback - DEBUG - Processing node 69bb4f0ca1e2bfb1b3347106: baseline-arm-baylibre 2026-03-21 01:19:18,668 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb4f0ca1e2bfb1b3347106 2026-03-21 01:19:18,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:19:18,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4f0ca1e2bfb1b3347106 HTTP/1.1" 200 None 2026-03-21 01:19:18,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:19:18,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-21 01:19:18,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:19:18,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb4f0ca1e2bfb1b3347106 HTTP/1.1" 200 None 2026-03-21 01:19:18,721 - lava_callback - ERROR - Error processing callback for node 69bb4f0ca1e2bfb1b3347106: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49840 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:20:12,411 - lava_callback - INFO - Received callback for node 69bb4f36a1e2bfb1b334710f INFO: 172.21.0.1:40870 - "POST /node/69bb4f36a1e2bfb1b334710f HTTP/1.0" 202 Accepted 2026-03-21 01:20:12,416 - lava_callback - INFO - Starting async job submit for node 69bb4f36a1e2bfb1b334710f 2026-03-21 01:20:12,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:20:12,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4f36a1e2bfb1b334710f HTTP/1.1" 200 None 2026-03-21 01:20:12,433 - lava_callback - DEBUG - Processing node 69bb4f36a1e2bfb1b334710f: baseline-arm-clabbe 2026-03-21 01:20:12,433 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb4f36a1e2bfb1b334710f 2026-03-21 01:20:12,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:20:12,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4f36a1e2bfb1b334710f HTTP/1.1" 200 None 2026-03-21 01:20:12,448 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:20:12,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-21 01:20:12,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:20:12,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb4f36a1e2bfb1b334710f HTTP/1.1" 200 None 2026-03-21 01:20:12,495 - lava_callback - ERROR - Error processing callback for node 69bb4f36a1e2bfb1b334710f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40878 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43004 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:20:32,430 - lava_callback - INFO - Received callback for node 69bb4f5ba1e2bfb1b3347119 INFO: 172.21.0.1:42504 - "POST /node/69bb4f5ba1e2bfb1b3347119 HTTP/1.0" 202 Accepted 2026-03-21 01:20:32,436 - lava_callback - INFO - Starting async job submit for node 69bb4f5ba1e2bfb1b3347119 2026-03-21 01:20:32,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:20:32,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4f5ba1e2bfb1b3347119 HTTP/1.1" 200 None 2026-03-21 01:20:32,455 - lava_callback - DEBUG - Processing node 69bb4f5ba1e2bfb1b3347119: baseline-arm-clabbe 2026-03-21 01:20:32,455 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb4f5ba1e2bfb1b3347119 2026-03-21 01:20:32,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:20:32,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4f5ba1e2bfb1b3347119 HTTP/1.1" 200 None 2026-03-21 01:20:32,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:20:32,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-21 01:20:32,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:20:32,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb4f5ba1e2bfb1b3347119 HTTP/1.1" 200 None 2026-03-21 01:20:32,518 - lava_callback - ERROR - Error processing callback for node 69bb4f5ba1e2bfb1b3347119: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36338 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55898 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:21:58,817 - lava_callback - INFO - Received callback for node 69bb4fa2a1e2bfb1b334712b INFO: 172.21.0.1:55902 - "POST /node/69bb4fa2a1e2bfb1b334712b HTTP/1.0" 202 Accepted 2026-03-21 01:21:58,823 - lava_callback - INFO - Starting async job submit for node 69bb4fa2a1e2bfb1b334712b 2026-03-21 01:21:58,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:21:58,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4fa2a1e2bfb1b334712b HTTP/1.1" 200 None 2026-03-21 01:21:58,838 - lava_callback - DEBUG - Processing node 69bb4fa2a1e2bfb1b334712b: baseline-arm-baylibre 2026-03-21 01:21:58,838 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb4fa2a1e2bfb1b334712b 2026-03-21 01:21:58,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:21:58,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4fa2a1e2bfb1b334712b HTTP/1.1" 200 None 2026-03-21 01:21:58,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:21:58,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-21 01:21:58,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:21:58,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb4fa2a1e2bfb1b334712b HTTP/1.1" 200 None 2026-03-21 01:21:58,904 - lava_callback - ERROR - Error processing callback for node 69bb4fa2a1e2bfb1b334712b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49316 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58292 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:22:58,911 - lava_callback - INFO - Received callback for node 69bb4feba1e2bfb1b3347140 INFO: 172.21.0.1:58300 - "POST /node/69bb4feba1e2bfb1b3347140 HTTP/1.0" 202 Accepted 2026-03-21 01:22:58,916 - lava_callback - INFO - Starting async job submit for node 69bb4feba1e2bfb1b3347140 2026-03-21 01:22:58,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:22:58,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4feba1e2bfb1b3347140 HTTP/1.1" 200 None 2026-03-21 01:22:58,932 - lava_callback - DEBUG - Processing node 69bb4feba1e2bfb1b3347140: baseline-arm-baylibre 2026-03-21 01:22:58,932 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb4feba1e2bfb1b3347140 2026-03-21 01:22:58,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:22:58,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb4feba1e2bfb1b3347140 HTTP/1.1" 200 None 2026-03-21 01:22:58,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:22:58,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-21 01:22:58,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:22:58,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb4feba1e2bfb1b3347140 HTTP/1.1" 200 None 2026-03-21 01:22:58,986 - lava_callback - ERROR - Error processing callback for node 69bb4feba1e2bfb1b3347140: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35074 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46964 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:23:29,986 - lava_callback - INFO - Received callback for node 69bdee80d54b5122738c67e5 INFO: 172.21.0.1:43690 - "POST /node/69bdee80d54b5122738c67e5 HTTP/1.0" 202 Accepted 2026-03-21 01:23:30,033 - lava_callback - INFO - Starting async job submit for node 69bdee80d54b5122738c67e5 2026-03-21 01:23:30,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:23:30,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdee80d54b5122738c67e5 HTTP/1.1" 200 None 2026-03-21 01:23:30,075 - lava_callback - DEBUG - Processing node 69bdee80d54b5122738c67e5: baseline-arm-pengutronix 2026-03-21 01:23:30,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:23:30,300 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:23:30,302 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69bdee80d54b5122738c67e5/log.txt.gz 2026-03-21 01:23:30,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:23:30,535 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:23:30,537 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69bdee80d54b5122738c67e5/lava-logs.yaml 2026-03-21 01:23:30,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:23:30,667 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:23:30,669 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69bdee80d54b5122738c67e5/lava_callback.json.gz DEBUG: Setup failed for 69bdee80d54b5122738c67e5. Transitting job node result: incomplete -> fail 2026-03-21 01:23:30,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:23:30,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdee80d54b5122738c67e5 HTTP/1.1" 200 None 2026-03-21 01:23:30,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:23:30,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-21 01:23:30,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:23:30,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdee80d54b5122738c67e5 HTTP/1.1" 200 None 2026-03-21 01:23:30,819 - lava_callback - INFO - Completed processing callback for node 69bdee80d54b5122738c67e5 2026-03-21 01:23:32,574 - lava_callback - INFO - Received callback for node 69bb5011a1e2bfb1b334714a INFO: 172.21.0.1:43694 - "POST /node/69bb5011a1e2bfb1b334714a HTTP/1.0" 202 Accepted 2026-03-21 01:23:32,580 - lava_callback - INFO - Starting async job submit for node 69bb5011a1e2bfb1b334714a 2026-03-21 01:23:32,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:23:32,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb5011a1e2bfb1b334714a HTTP/1.1" 200 None 2026-03-21 01:23:32,595 - lava_callback - DEBUG - Processing node 69bb5011a1e2bfb1b334714a: baseline-arm-clabbe 2026-03-21 01:23:32,595 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb5011a1e2bfb1b334714a 2026-03-21 01:23:32,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:23:32,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb5011a1e2bfb1b334714a HTTP/1.1" 200 None 2026-03-21 01:23:32,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:23:32,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-21 01:23:32,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:23:32,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb5011a1e2bfb1b334714a HTTP/1.1" 200 None 2026-03-21 01:23:32,654 - lava_callback - ERROR - Error processing callback for node 69bb5011a1e2bfb1b334714a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-21 01:23:36,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:23:36,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:43700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38864 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38874 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:24:18,942 - lava_callback - INFO - Received callback for node 69bb5034a1e2bfb1b3347157 INFO: 172.21.0.1:38880 - "POST /node/69bb5034a1e2bfb1b3347157 HTTP/1.0" 202 Accepted 2026-03-21 01:24:18,947 - lava_callback - INFO - Starting async job submit for node 69bb5034a1e2bfb1b3347157 2026-03-21 01:24:18,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:24:18,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb5034a1e2bfb1b3347157 HTTP/1.1" 200 None 2026-03-21 01:24:18,971 - lava_callback - DEBUG - Processing node 69bb5034a1e2bfb1b3347157: baseline-arm-baylibre 2026-03-21 01:24:18,972 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb5034a1e2bfb1b3347157 2026-03-21 01:24:18,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:24:18,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb5034a1e2bfb1b3347157 HTTP/1.1" 200 None 2026-03-21 01:24:18,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:24:19,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-21 01:24:19,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:24:19,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb5034a1e2bfb1b3347157 HTTP/1.1" 200 None 2026-03-21 01:24:19,041 - lava_callback - ERROR - Error processing callback for node 69bb5034a1e2bfb1b3347157: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36546 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:24:40,449 - lava_callback - INFO - Received callback for node 69bd7aea0e93e2314723e76a INFO: 172.21.0.1:52666 - "POST /node/69bd7aea0e93e2314723e76a HTTP/1.0" 202 Accepted 2026-03-21 01:24:40,456 - lava_callback - INFO - Starting async job submit for node 69bd7aea0e93e2314723e76a 2026-03-21 01:24:40,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:24:40,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd7aea0e93e2314723e76a HTTP/1.1" 200 None 2026-03-21 01:24:40,500 - lava_callback - DEBUG - Processing node 69bd7aea0e93e2314723e76a: baseline-arm64-broonie 2026-03-21 01:24:40,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:24:40,923 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:24:40,925 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69bd7aea0e93e2314723e76a/log.txt.gz 2026-03-21 01:24:41,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:24:41,311 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:24:41,313 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69bd7aea0e93e2314723e76a/lava-logs.yaml 2026-03-21 01:24:41,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:24:41,474 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:24:41,475 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69bd7aea0e93e2314723e76a/lava_callback.json.gz 2026-03-21 01:24:41,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:24:41,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd7aea0e93e2314723e76a HTTP/1.1" 200 None 2026-03-21 01:24:41,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:24:41,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-21 01:24:41,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:24:41,563 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd7aea0e93e2314723e76a HTTP/1.1" 200 None 2026-03-21 01:24:41,565 - lava_callback - INFO - Completed processing callback for node 69bd7aea0e93e2314723e76a INFO: 172.21.0.1:52682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56872 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:25:00,013 - lava_callback - INFO - Received callback for node 69bdd6bd0e93e23147242d11 INFO: 172.21.0.1:44112 - "POST /node/69bdd6bd0e93e23147242d11 HTTP/1.0" 202 Accepted 2026-03-21 01:25:00,022 - lava_callback - INFO - Starting async job submit for node 69bdd6bd0e93e23147242d11 2026-03-21 01:25:00,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:25:00,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd6bd0e93e23147242d11 HTTP/1.1" 200 None 2026-03-21 01:25:00,064 - lava_callback - DEBUG - Processing node 69bdd6bd0e93e23147242d11: baseline-arm64-broonie 2026-03-21 01:25:00,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:25:00,458 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:25:00,459 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69bdd6bd0e93e23147242d11/log.txt.gz 2026-03-21 01:25:00,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:25:00,825 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:25:00,827 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69bdd6bd0e93e23147242d11/lava-logs.yaml 2026-03-21 01:25:00,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:25:00,962 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:25:00,964 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69bdd6bd0e93e23147242d11/lava_callback.json.gz 2026-03-21 01:25:00,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:25:01,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd6bd0e93e23147242d11 HTTP/1.1" 200 None 2026-03-21 01:25:01,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:25:01,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-21 01:25:01,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:25:01,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdd6bd0e93e23147242d11 HTTP/1.1" 200 None 2026-03-21 01:25:01,052 - lava_callback - INFO - Completed processing callback for node 69bdd6bd0e93e23147242d11 INFO: 172.21.0.1:44116 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:25:06,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:25:06,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:44132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60510 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55518 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54664 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59408 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50960 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36936 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:30:12,731 - lava_callback - INFO - Received callback for node 69bb51a1a1e2bfb1b33471ae INFO: 172.21.0.1:53638 - "POST /node/69bb51a1a1e2bfb1b33471ae HTTP/1.0" 202 Accepted 2026-03-21 01:30:12,738 - lava_callback - INFO - Starting async job submit for node 69bb51a1a1e2bfb1b33471ae 2026-03-21 01:30:12,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:30:12,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb51a1a1e2bfb1b33471ae HTTP/1.1" 200 None 2026-03-21 01:30:12,762 - lava_callback - DEBUG - Processing node 69bb51a1a1e2bfb1b33471ae: baseline-arm-clabbe 2026-03-21 01:30:12,762 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb51a1a1e2bfb1b33471ae 2026-03-21 01:30:12,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:30:12,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb51a1a1e2bfb1b33471ae HTTP/1.1" 200 None 2026-03-21 01:30:12,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:30:12,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-21 01:30:12,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:30:12,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb51a1a1e2bfb1b33471ae HTTP/1.1" 200 None 2026-03-21 01:30:12,822 - lava_callback - ERROR - Error processing callback for node 69bb51a1a1e2bfb1b33471ae: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53640 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53644 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:30:25,256 - lava_callback - INFO - Received callback for node 69bdf489d54b5122738c69a4 INFO: 172.21.0.1:53300 - "POST /node/69bdf489d54b5122738c69a4 HTTP/1.0" 202 Accepted 2026-03-21 01:30:25,262 - lava_callback - INFO - Starting async job submit for node 69bdf489d54b5122738c69a4 2026-03-21 01:30:25,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:30:25,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf489d54b5122738c69a4 HTTP/1.1" 200 None 2026-03-21 01:30:25,302 - lava_callback - DEBUG - Processing node 69bdf489d54b5122738c69a4: baseline-arm64 2026-03-21 01:30:25,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:30:25,641 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:30:25,643 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdf489d54b5122738c69a4/log.txt.gz 2026-03-21 01:30:25,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:30:25,951 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:30:25,953 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdf489d54b5122738c69a4/lava-logs.yaml 2026-03-21 01:30:25,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:30:26,116 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:30:26,118 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdf489d54b5122738c69a4/lava_callback.json.gz 2026-03-21 01:30:26,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:30:26,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf489d54b5122738c69a4 HTTP/1.1" 200 None 2026-03-21 01:30:26,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:30:26,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-21 01:30:26,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:30:26,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdf489d54b5122738c69a4 HTTP/1.1" 200 None 2026-03-21 01:30:26,199 - lava_callback - INFO - Completed processing callback for node 69bdf489d54b5122738c69a4 INFO: 172.21.0.1:53308 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:30:31,476 - lava_callback - INFO - Received callback for node 69bdf4a7d54b5122738c69f9 INFO: 172.21.0.1:37092 - "POST /node/69bdf4a7d54b5122738c69f9 HTTP/1.0" 202 Accepted 2026-03-21 01:30:31,482 - lava_callback - INFO - Starting async job submit for node 69bdf4a7d54b5122738c69f9 2026-03-21 01:30:31,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:30:31,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf4a7d54b5122738c69f9 HTTP/1.1" 200 None 2026-03-21 01:30:31,503 - lava_callback - DEBUG - Processing node 69bdf4a7d54b5122738c69f9: baseline-nfs-arm64 2026-03-21 01:30:31,505 - lava_callback - WARNING - Failed to upload log 2026-03-21 01:30:31,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:30:31,665 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:30:31,667 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69bdf4a7d54b5122738c69f9/lava-logs.yaml 2026-03-21 01:30:31,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:30:31,841 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:30:31,843 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69bdf4a7d54b5122738c69f9/lava_callback.json.gz 2026-03-21 01:30:31,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:30:31,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf4a7d54b5122738c69f9 HTTP/1.1" 200 None 2026-03-21 01:30:31,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:30:31,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-21 01:30:31,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:30:31,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdf4a7d54b5122738c69f9 HTTP/1.1" 200 None 2026-03-21 01:30:31,928 - lava_callback - INFO - Completed processing callback for node 69bdf4a7d54b5122738c69f9 2026-03-21 01:30:34,349 - lava_callback - INFO - Received callback for node 69bdf495d54b5122738c69d0 INFO: 172.21.0.1:37108 - "POST /node/69bdf495d54b5122738c69d0 HTTP/1.0" 202 Accepted 2026-03-21 01:30:34,356 - lava_callback - INFO - Starting async job submit for node 69bdf495d54b5122738c69d0 2026-03-21 01:30:34,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:30:34,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf495d54b5122738c69d0 HTTP/1.1" 200 None 2026-03-21 01:30:34,403 - lava_callback - DEBUG - Processing node 69bdf495d54b5122738c69d0: baseline-arm64 2026-03-21 01:30:34,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:30:34,666 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:30:34,669 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdf495d54b5122738c69d0/log.txt.gz 2026-03-21 01:30:34,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:30:34,969 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:30:34,971 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdf495d54b5122738c69d0/lava-logs.yaml 2026-03-21 01:30:34,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:30:35,220 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:30:35,222 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdf495d54b5122738c69d0/lava_callback.json.gz 2026-03-21 01:30:35,240 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:30:35,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf495d54b5122738c69d0 HTTP/1.1" 200 None 2026-03-21 01:30:35,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:30:35,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-21 01:30:35,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:30:35,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdf495d54b5122738c69d0 HTTP/1.1" 200 None 2026-03-21 01:30:35,302 - lava_callback - INFO - Completed processing callback for node 69bdf495d54b5122738c69d0 2026-03-21 01:30:36,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:30:36,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:37124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48490 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:30:56,270 - lava_callback - INFO - Received callback for node 69bdf48dd54b5122738c69b0 INFO: 172.21.0.1:47904 - "POST /node/69bdf48dd54b5122738c69b0 HTTP/1.0" 202 Accepted 2026-03-21 01:30:56,277 - lava_callback - INFO - Starting async job submit for node 69bdf48dd54b5122738c69b0 2026-03-21 01:30:56,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:30:56,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf48dd54b5122738c69b0 HTTP/1.1" 200 None 2026-03-21 01:30:56,324 - lava_callback - DEBUG - Processing node 69bdf48dd54b5122738c69b0: baseline-arm64 2026-03-21 01:30:56,486 - lava_callback - WARNING - Failed to upload log INFO: 172.21.0.1:47914 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:30:56,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:30:56,655 - lava_callback - INFO - Received callback for node 69bdf48ad54b5122738c69aa INFO: 172.21.0.1:47924 - "POST /node/69bdf48ad54b5122738c69aa HTTP/1.0" 202 Accepted 2026-03-21 01:30:56,662 - lava_callback - INFO - Starting async job submit for node 69bdf48ad54b5122738c69aa 2026-03-21 01:30:56,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:30:56,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf48ad54b5122738c69aa HTTP/1.1" 200 None 2026-03-21 01:30:56,700 - lava_callback - DEBUG - Processing node 69bdf48ad54b5122738c69aa: baseline-arm64 2026-03-21 01:30:56,748 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:30:56,800 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdf48dd54b5122738c69b0/lava-logs.yaml 2026-03-21 01:30:56,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:30:56,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:30:57,146 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:30:57,148 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdf48dd54b5122738c69b0/lava_callback.json.gz 2026-03-21 01:30:57,182 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:30:57,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:30:57,184 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdf48ad54b5122738c69aa/log.txt.gz 2026-03-21 01:30:57,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf48dd54b5122738c69b0 HTTP/1.1" 200 None 2026-03-21 01:30:57,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:30:57,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-21 01:30:57,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:30:57,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:30:57,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdf48dd54b5122738c69b0 HTTP/1.1" 200 None 2026-03-21 01:30:57,497 - lava_callback - INFO - Completed processing callback for node 69bdf48dd54b5122738c69b0 2026-03-21 01:30:57,662 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:30:57,664 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdf48ad54b5122738c69aa/lava-logs.yaml 2026-03-21 01:30:57,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:30:57,844 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:30:57,846 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdf48ad54b5122738c69aa/lava_callback.json.gz 2026-03-21 01:30:57,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:30:57,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf48ad54b5122738c69aa HTTP/1.1" 200 None 2026-03-21 01:30:57,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:30:57,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-21 01:30:57,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:30:57,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdf48ad54b5122738c69aa HTTP/1.1" 200 None 2026-03-21 01:30:57,946 - lava_callback - INFO - Completed processing callback for node 69bdf48ad54b5122738c69aa 2026-03-21 01:30:58,466 - lava_callback - INFO - Received callback for node 69bdf491d54b5122738c69c7 INFO: 172.21.0.1:47932 - "POST /node/69bdf491d54b5122738c69c7 HTTP/1.0" 202 Accepted 2026-03-21 01:30:58,473 - lava_callback - INFO - Starting async job submit for node 69bdf491d54b5122738c69c7 2026-03-21 01:30:58,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:30:58,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf491d54b5122738c69c7 HTTP/1.1" 200 None 2026-03-21 01:30:58,509 - lava_callback - DEBUG - Processing node 69bdf491d54b5122738c69c7: baseline-arm64 2026-03-21 01:30:58,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:30:59,024 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:30:59,025 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdf491d54b5122738c69c7/log.txt.gz 2026-03-21 01:30:59,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:30:59,630 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:30:59,632 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdf491d54b5122738c69c7/lava-logs.yaml 2026-03-21 01:30:59,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:30:59,832 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:30:59,834 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdf491d54b5122738c69c7/lava_callback.json.gz 2026-03-21 01:30:59,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:30:59,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf491d54b5122738c69c7 HTTP/1.1" 200 None 2026-03-21 01:30:59,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:30:59,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-21 01:30:59,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:30:59,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdf491d54b5122738c69c7 HTTP/1.1" 200 None 2026-03-21 01:30:59,918 - lava_callback - INFO - Completed processing callback for node 69bdf491d54b5122738c69c7 2026-03-21 01:31:00,369 - lava_callback - INFO - Received callback for node 69bdf493d54b5122738c69ca INFO: 172.21.0.1:55546 - "POST /node/69bdf493d54b5122738c69ca HTTP/1.0" 202 Accepted 2026-03-21 01:31:00,380 - lava_callback - INFO - Starting async job submit for node 69bdf493d54b5122738c69ca 2026-03-21 01:31:00,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:31:00,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf493d54b5122738c69ca HTTP/1.1" 200 None 2026-03-21 01:31:00,424 - lava_callback - DEBUG - Processing node 69bdf493d54b5122738c69ca: baseline-arm64 2026-03-21 01:31:00,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:31:01,034 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:31:01,036 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdf493d54b5122738c69ca/log.txt.gz 2026-03-21 01:31:01,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:31:01,564 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:31:01,566 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdf493d54b5122738c69ca/lava-logs.yaml 2026-03-21 01:31:01,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:31:01,740 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:31:01,742 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdf493d54b5122738c69ca/lava_callback.json.gz 2026-03-21 01:31:01,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:31:01,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf493d54b5122738c69ca HTTP/1.1" 200 None 2026-03-21 01:31:01,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:31:01,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-21 01:31:01,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:31:01,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdf493d54b5122738c69ca HTTP/1.1" 200 None 2026-03-21 01:31:01,831 - lava_callback - INFO - Completed processing callback for node 69bdf493d54b5122738c69ca INFO: 172.21.0.1:55554 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:31:05,181 - lava_callback - INFO - Received callback for node 69bdf48ed54b5122738c69be INFO: 172.21.0.1:55556 - "POST /node/69bdf48ed54b5122738c69be HTTP/1.0" 202 Accepted 2026-03-21 01:31:05,188 - lava_callback - INFO - Starting async job submit for node 69bdf48ed54b5122738c69be 2026-03-21 01:31:05,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:31:05,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf48ed54b5122738c69be HTTP/1.1" 200 None 2026-03-21 01:31:05,233 - lava_callback - DEBUG - Processing node 69bdf48ed54b5122738c69be: baseline-arm64 2026-03-21 01:31:05,473 - lava_callback - WARNING - Failed to upload log 2026-03-21 01:31:05,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:31:05,768 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:31:05,770 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdf48ed54b5122738c69be/lava-logs.yaml 2026-03-21 01:31:05,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:31:05,896 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:31:05,898 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdf48ed54b5122738c69be/lava_callback.json.gz 2026-03-21 01:31:05,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:31:05,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf48ed54b5122738c69be HTTP/1.1" 200 None 2026-03-21 01:31:05,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:31:05,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-21 01:31:05,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:31:05,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdf48ed54b5122738c69be HTTP/1.1" 200 None 2026-03-21 01:31:06,069 - lava_callback - INFO - Completed processing callback for node 69bdf48ed54b5122738c69be 2026-03-21 01:31:06,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:31:06,157 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.21.0.1:55562 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:31:13,157 - lava_callback - INFO - Received callback for node 69bdf49bd54b5122738c69dd INFO: 172.21.0.1:38502 - "POST /node/69bdf49bd54b5122738c69dd HTTP/1.0" 202 Accepted 2026-03-21 01:31:13,164 - lava_callback - INFO - Starting async job submit for node 69bdf49bd54b5122738c69dd 2026-03-21 01:31:13,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:31:13,201 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf49bd54b5122738c69dd HTTP/1.1" 200 None 2026-03-21 01:31:13,202 - lava_callback - DEBUG - Processing node 69bdf49bd54b5122738c69dd: baseline-arm64 2026-03-21 01:31:13,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:31:13,561 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:31:13,562 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdf49bd54b5122738c69dd/log.txt.gz 2026-03-21 01:31:13,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:31:13,914 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:31:13,916 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdf49bd54b5122738c69dd/lava-logs.yaml 2026-03-21 01:31:13,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:31:14,061 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:31:14,063 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdf49bd54b5122738c69dd/lava_callback.json.gz 2026-03-21 01:31:14,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:31:14,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf49bd54b5122738c69dd HTTP/1.1" 200 None 2026-03-21 01:31:14,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:31:14,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-21 01:31:14,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:31:14,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdf49bd54b5122738c69dd HTTP/1.1" 200 None 2026-03-21 01:31:14,146 - lava_callback - INFO - Completed processing callback for node 69bdf49bd54b5122738c69dd INFO: 172.21.0.1:38514 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-21 01:31:16,524 - lava_callback - INFO - Received callback for node 69bdf49cd54b5122738c69e0 INFO: 172.21.0.1:38516 - "POST /node/69bdf49cd54b5122738c69e0 HTTP/1.0" 202 Accepted 2026-03-21 01:31:16,531 - lava_callback - INFO - Starting async job submit for node 69bdf49cd54b5122738c69e0 INFO: 172.21.0.1:38530 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:31:16,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:31:16,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf49cd54b5122738c69e0 HTTP/1.1" 200 None 2026-03-21 01:31:16,568 - lava_callback - DEBUG - Processing node 69bdf49cd54b5122738c69e0: baseline-arm64 2026-03-21 01:31:16,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:31:16,992 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:31:16,994 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdf49cd54b5122738c69e0/log.txt.gz 2026-03-21 01:31:17,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:31:17,373 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:31:17,375 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdf49cd54b5122738c69e0/lava-logs.yaml 2026-03-21 01:31:17,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:31:17,528 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:31:17,530 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdf49cd54b5122738c69e0/lava_callback.json.gz 2026-03-21 01:31:17,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:31:17,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf49cd54b5122738c69e0 HTTP/1.1" 200 None 2026-03-21 01:31:17,573 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:31:17,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-21 01:31:17,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:31:17,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdf49cd54b5122738c69e0 HTTP/1.1" 200 None 2026-03-21 01:31:17,639 - lava_callback - INFO - Completed processing callback for node 69bdf49cd54b5122738c69e0 2026-03-21 01:31:18,117 - lava_callback - INFO - Received callback for node 69bdf496d54b5122738c69d3 INFO: 172.21.0.1:38538 - "POST /node/69bdf496d54b5122738c69d3 HTTP/1.0" 202 Accepted 2026-03-21 01:31:18,124 - lava_callback - INFO - Starting async job submit for node 69bdf496d54b5122738c69d3 2026-03-21 01:31:18,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:31:18,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf496d54b5122738c69d3 HTTP/1.1" 200 None 2026-03-21 01:31:18,164 - lava_callback - DEBUG - Processing node 69bdf496d54b5122738c69d3: baseline-arm64 2026-03-21 01:31:18,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:31:18,515 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:31:18,516 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdf496d54b5122738c69d3/log.txt.gz 2026-03-21 01:31:18,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:31:18,816 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:31:18,818 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdf496d54b5122738c69d3/lava-logs.yaml 2026-03-21 01:31:18,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:31:18,963 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:31:18,965 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdf496d54b5122738c69d3/lava_callback.json.gz 2026-03-21 01:31:18,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:31:19,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf496d54b5122738c69d3 HTTP/1.1" 200 None 2026-03-21 01:31:19,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:31:19,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-21 01:31:19,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:31:19,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdf496d54b5122738c69d3 HTTP/1.1" 200 None 2026-03-21 01:31:19,054 - lava_callback - INFO - Completed processing callback for node 69bdf496d54b5122738c69d3 2026-03-21 01:31:21,679 - lava_callback - INFO - Received callback for node 69bdf489d54b5122738c69a7 INFO: 172.21.0.1:51698 - "POST /node/69bdf489d54b5122738c69a7 HTTP/1.0" 202 Accepted 2026-03-21 01:31:21,685 - lava_callback - INFO - Starting async job submit for node 69bdf489d54b5122738c69a7 2026-03-21 01:31:21,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:31:21,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf489d54b5122738c69a7 HTTP/1.1" 200 None 2026-03-21 01:31:21,725 - lava_callback - DEBUG - Processing node 69bdf489d54b5122738c69a7: baseline-arm64 2026-03-21 01:31:21,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:31:22,060 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:31:22,062 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdf489d54b5122738c69a7/log.txt.gz 2026-03-21 01:31:22,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:31:22,359 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:31:22,361 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdf489d54b5122738c69a7/lava-logs.yaml 2026-03-21 01:31:22,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:31:22,505 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:31:22,506 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdf489d54b5122738c69a7/lava_callback.json.gz 2026-03-21 01:31:22,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:31:22,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf489d54b5122738c69a7 HTTP/1.1" 200 None 2026-03-21 01:31:22,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:31:22,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-21 01:31:22,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:31:22,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdf489d54b5122738c69a7 HTTP/1.1" 200 None 2026-03-21 01:31:22,585 - lava_callback - INFO - Completed processing callback for node 69bdf489d54b5122738c69a7 2026-03-21 01:31:24,106 - lava_callback - INFO - Received callback for node 69bdf490d54b5122738c69c4 INFO: 172.21.0.1:51714 - "POST /node/69bdf490d54b5122738c69c4 HTTP/1.0" 202 Accepted 2026-03-21 01:31:24,112 - lava_callback - INFO - Starting async job submit for node 69bdf490d54b5122738c69c4 2026-03-21 01:31:24,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:31:24,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf490d54b5122738c69c4 HTTP/1.1" 200 None 2026-03-21 01:31:24,149 - lava_callback - DEBUG - Processing node 69bdf490d54b5122738c69c4: baseline-arm64 2026-03-21 01:31:24,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:31:24,691 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:31:24,692 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdf490d54b5122738c69c4/log.txt.gz 2026-03-21 01:31:24,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:31:25,016 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:31:25,018 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdf490d54b5122738c69c4/lava-logs.yaml 2026-03-21 01:31:25,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:31:25,159 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:31:25,160 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdf490d54b5122738c69c4/lava_callback.json.gz 2026-03-21 01:31:25,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:31:25,203 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf490d54b5122738c69c4 HTTP/1.1" 200 None 2026-03-21 01:31:25,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:31:25,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-21 01:31:25,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:31:25,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdf490d54b5122738c69c4 HTTP/1.1" 200 None 2026-03-21 01:31:25,252 - lava_callback - INFO - Completed processing callback for node 69bdf490d54b5122738c69c4 INFO: 172.21.0.1:51724 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:31:28,336 - lava_callback - INFO - Received callback for node 69bdf4acd54b5122738c6a1f INFO: 172.21.0.1:51734 - "POST /node/69bdf4acd54b5122738c6a1f HTTP/1.0" 202 Accepted 2026-03-21 01:31:28,343 - lava_callback - INFO - Starting async job submit for node 69bdf4acd54b5122738c6a1f 2026-03-21 01:31:28,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:31:28,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf4acd54b5122738c6a1f HTTP/1.1" 200 None 2026-03-21 01:31:28,386 - lava_callback - DEBUG - Processing node 69bdf4acd54b5122738c6a1f: baseline-nfs-arm64 2026-03-21 01:31:28,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:31:28,838 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:31:28,839 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69bdf4acd54b5122738c6a1f/log.txt.gz 2026-03-21 01:31:29,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:31:29,310 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:31:29,312 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69bdf4acd54b5122738c6a1f/lava-logs.yaml 2026-03-21 01:31:29,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:31:29,459 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:31:29,461 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69bdf4acd54b5122738c6a1f/lava_callback.json.gz 2026-03-21 01:31:29,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:31:29,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf4acd54b5122738c6a1f HTTP/1.1" 200 None 2026-03-21 01:31:29,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:31:29,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-21 01:31:29,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:31:29,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdf4acd54b5122738c6a1f HTTP/1.1" 200 None 2026-03-21 01:31:29,541 - lava_callback - INFO - Completed processing callback for node 69bdf4acd54b5122738c6a1f 2026-03-21 01:31:31,038 - lava_callback - INFO - Received callback for node 69bdf48bd54b5122738c69ad INFO: 172.21.0.1:53906 - "POST /node/69bdf48bd54b5122738c69ad HTTP/1.0" 202 Accepted 2026-03-21 01:31:31,044 - lava_callback - INFO - Starting async job submit for node 69bdf48bd54b5122738c69ad 2026-03-21 01:31:31,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:31:31,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf48bd54b5122738c69ad HTTP/1.1" 200 None 2026-03-21 01:31:31,103 - lava_callback - DEBUG - Processing node 69bdf48bd54b5122738c69ad: baseline-arm64 2026-03-21 01:31:31,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:31:31,480 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:31:31,481 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdf48bd54b5122738c69ad/log.txt.gz 2026-03-21 01:31:31,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:31:31,808 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:31:31,810 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdf48bd54b5122738c69ad/lava-logs.yaml 2026-03-21 01:31:31,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:31:31,982 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:31:31,983 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdf48bd54b5122738c69ad/lava_callback.json.gz 2026-03-21 01:31:32,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:31:32,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf48bd54b5122738c69ad HTTP/1.1" 200 None 2026-03-21 01:31:32,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:31:32,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-21 01:31:32,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:31:32,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdf48bd54b5122738c69ad HTTP/1.1" 200 None 2026-03-21 01:31:32,077 - lava_callback - INFO - Completed processing callback for node 69bdf48bd54b5122738c69ad 2026-03-21 01:31:36,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:31:36,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.21.0.1:53910 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:31:37,837 - lava_callback - INFO - Received callback for node 69bdf4abd54b5122738c6a03 INFO: 172.21.0.1:53922 - "POST /node/69bdf4abd54b5122738c6a03 HTTP/1.0" 202 Accepted 2026-03-21 01:31:37,844 - lava_callback - INFO - Starting async job submit for node 69bdf4abd54b5122738c6a03 2026-03-21 01:31:37,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:31:37,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf4abd54b5122738c6a03 HTTP/1.1" 200 None 2026-03-21 01:31:37,890 - lava_callback - DEBUG - Processing node 69bdf4abd54b5122738c6a03: baseline-nfs-arm64 2026-03-21 01:31:38,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:31:38,373 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:31:38,375 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69bdf4abd54b5122738c6a03/log.txt.gz 2026-03-21 01:31:38,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:31:38,788 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:31:38,790 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69bdf4abd54b5122738c6a03/lava-logs.yaml 2026-03-21 01:31:38,798 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:31:38,957 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:31:38,959 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69bdf4abd54b5122738c6a03/lava_callback.json.gz 2026-03-21 01:31:38,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:31:38,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf4abd54b5122738c6a03 HTTP/1.1" 200 None 2026-03-21 01:31:38,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:31:39,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-21 01:31:39,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:31:39,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdf4abd54b5122738c6a03 HTTP/1.1" 200 None 2026-03-21 01:31:39,040 - lava_callback - INFO - Completed processing callback for node 69bdf4abd54b5122738c6a03 2026-03-21 01:31:44,484 - lava_callback - INFO - Received callback for node 69bdf499d54b5122738c69da INFO: 172.21.0.1:56350 - "POST /node/69bdf499d54b5122738c69da HTTP/1.0" 202 Accepted 2026-03-21 01:31:44,490 - lava_callback - INFO - Starting async job submit for node 69bdf499d54b5122738c69da 2026-03-21 01:31:44,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:31:44,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf499d54b5122738c69da HTTP/1.1" 200 None 2026-03-21 01:31:44,526 - lava_callback - DEBUG - Processing node 69bdf499d54b5122738c69da: baseline-arm64 2026-03-21 01:31:44,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:31:44,927 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:31:44,928 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdf499d54b5122738c69da/log.txt.gz 2026-03-21 01:31:45,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:31:45,264 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:31:45,266 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdf499d54b5122738c69da/lava-logs.yaml 2026-03-21 01:31:45,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:31:45,413 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:31:45,415 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdf499d54b5122738c69da/lava_callback.json.gz 2026-03-21 01:31:45,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:31:45,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf499d54b5122738c69da HTTP/1.1" 200 None 2026-03-21 01:31:45,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:31:45,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-21 01:31:45,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:31:45,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdf499d54b5122738c69da HTTP/1.1" 200 None 2026-03-21 01:31:45,506 - lava_callback - INFO - Completed processing callback for node 69bdf499d54b5122738c69da 2026-03-21 01:31:45,559 - lava_callback - INFO - Received callback for node 69bdf4a2d54b5122738c69ec INFO: 172.21.0.1:56358 - "POST /node/69bdf4a2d54b5122738c69ec HTTP/1.0" 202 Accepted 2026-03-21 01:31:45,570 - lava_callback - INFO - Starting async job submit for node 69bdf4a2d54b5122738c69ec 2026-03-21 01:31:45,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:31:45,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf4a2d54b5122738c69ec HTTP/1.1" 200 None 2026-03-21 01:31:45,625 - lava_callback - DEBUG - Processing node 69bdf4a2d54b5122738c69ec: baseline-nfs-arm64 2026-03-21 01:31:46,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:31:46,142 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:31:46,144 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69bdf4a2d54b5122738c69ec/log.txt.gz 2026-03-21 01:31:46,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:56368 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:31:46,529 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:31:46,531 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69bdf4a2d54b5122738c69ec/lava-logs.yaml 2026-03-21 01:31:46,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:31:46,669 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:31:46,671 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69bdf4a2d54b5122738c69ec/lava_callback.json.gz 2026-03-21 01:31:46,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:31:46,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf4a2d54b5122738c69ec HTTP/1.1" 200 None 2026-03-21 01:31:46,711 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:31:46,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-21 01:31:46,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:31:46,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdf4a2d54b5122738c69ec HTTP/1.1" 200 None 2026-03-21 01:31:46,755 - lava_callback - INFO - Completed processing callback for node 69bdf4a2d54b5122738c69ec 2026-03-21 01:31:49,710 - lava_callback - INFO - Received callback for node 69bdf4a5d54b5122738c69f3 INFO: 172.21.0.1:52904 - "POST /node/69bdf4a5d54b5122738c69f3 HTTP/1.0" 202 Accepted 2026-03-21 01:31:49,717 - lava_callback - INFO - Starting async job submit for node 69bdf4a5d54b5122738c69f3 2026-03-21 01:31:49,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:31:49,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf4a5d54b5122738c69f3 HTTP/1.1" 200 None 2026-03-21 01:31:49,773 - lava_callback - DEBUG - Processing node 69bdf4a5d54b5122738c69f3: baseline-nfs-arm64 2026-03-21 01:31:50,204 - lava_callback - INFO - Received callback for node 69bdf4bad54b5122738c6a41 INFO: 172.21.0.1:52918 - "POST /node/69bdf4bad54b5122738c6a41 HTTP/1.0" 202 Accepted 2026-03-21 01:31:50,212 - lava_callback - INFO - Starting async job submit for node 69bdf4bad54b5122738c6a41 2026-03-21 01:31:50,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:31:50,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:31:50,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf4bad54b5122738c6a41 HTTP/1.1" 200 None 2026-03-21 01:31:50,262 - lava_callback - DEBUG - Processing node 69bdf4bad54b5122738c6a41: baseline-arm64-broonie 2026-03-21 01:31:50,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:31:50,527 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:31:50,529 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69bdf4a5d54b5122738c69f3/log.txt.gz 2026-03-21 01:31:50,735 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:31:50,785 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69bdf4bad54b5122738c6a41/log.txt.gz 2026-03-21 01:31:50,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:31:50,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:31:51,180 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:31:51,182 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69bdf4a5d54b5122738c69f3/lava-logs.yaml 2026-03-21 01:31:51,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:31:51,198 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:31:51,200 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69bdf4bad54b5122738c6a41/lava-logs.yaml 2026-03-21 01:31:51,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:31:51,331 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:31:51,333 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69bdf4bad54b5122738c6a41/lava_callback.json.gz 2026-03-21 01:31:51,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:31:51,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf4bad54b5122738c6a41 HTTP/1.1" 200 None 2026-03-21 01:31:51,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:31:51,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-21 01:31:51,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:31:51,422 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:31:51,424 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69bdf4a5d54b5122738c69f3/lava_callback.json.gz 2026-03-21 01:31:51,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdf4bad54b5122738c6a41 HTTP/1.1" 200 None 2026-03-21 01:31:51,459 - lava_callback - INFO - Completed processing callback for node 69bdf4bad54b5122738c6a41 2026-03-21 01:31:51,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:31:51,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf4a5d54b5122738c69f3 HTTP/1.1" 200 None 2026-03-21 01:31:51,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:31:51,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-21 01:31:51,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:31:51,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdf4a5d54b5122738c69f3 HTTP/1.1" 200 None 2026-03-21 01:31:51,590 - lava_callback - INFO - Completed processing callback for node 69bdf4a5d54b5122738c69f3 2026-03-21 01:31:53,738 - lava_callback - INFO - Received callback for node 69bdf4d4d54b5122738c6ab1 INFO: 172.21.0.1:52928 - "POST /node/69bdf4d4d54b5122738c6ab1 HTTP/1.0" 202 Accepted 2026-03-21 01:31:53,748 - lava_callback - INFO - Starting async job submit for node 69bdf4d4d54b5122738c6ab1 2026-03-21 01:31:53,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:31:53,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf4d4d54b5122738c6ab1 HTTP/1.1" 200 None 2026-03-21 01:31:53,785 - lava_callback - DEBUG - Processing node 69bdf4d4d54b5122738c6ab1: baseline-arm64-foundriesio 2026-03-21 01:31:53,862 - lava_callback - INFO - Received callback for node 69bdf4a8d54b5122738c69fd INFO: 172.21.0.1:52940 - "POST /node/69bdf4a8d54b5122738c69fd HTTP/1.0" 202 Accepted 2026-03-21 01:31:53,890 - lava_callback - INFO - Starting async job submit for node 69bdf4a8d54b5122738c69fd 2026-03-21 01:31:53,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:31:54,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf4a8d54b5122738c69fd HTTP/1.1" 200 None 2026-03-21 01:31:54,037 - lava_callback - DEBUG - Processing node 69bdf4a8d54b5122738c69fd: baseline-nfs-arm64 2026-03-21 01:31:54,119 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:31:54,325 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:31:54,358 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69bdf4d4d54b5122738c6ab1/log.txt.gz 2026-03-21 01:31:54,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:31:54,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:31:54,726 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:31:54,728 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69bdf4a8d54b5122738c69fd/log.txt.gz 2026-03-21 01:31:54,849 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:31:54,880 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69bdf4d4d54b5122738c6ab1/lava-logs.yaml 2026-03-21 01:31:54,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:31:55,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:31:55,152 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:31:55,154 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69bdf4d4d54b5122738c6ab1/lava_callback.json.gz 2026-03-21 01:31:55,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:31:55,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf4d4d54b5122738c6ab1 HTTP/1.1" 200 None 2026-03-21 01:31:55,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:31:55,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-21 01:31:55,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:31:55,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdf4d4d54b5122738c6ab1 HTTP/1.1" 200 None 2026-03-21 01:31:55,254 - lava_callback - INFO - Completed processing callback for node 69bdf4d4d54b5122738c6ab1 2026-03-21 01:31:55,280 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:31:55,283 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69bdf4a8d54b5122738c69fd/lava-logs.yaml 2026-03-21 01:31:55,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:31:55,510 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:31:55,512 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69bdf4a8d54b5122738c69fd/lava_callback.json.gz 2026-03-21 01:31:55,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:31:55,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf4a8d54b5122738c69fd HTTP/1.1" 200 None 2026-03-21 01:31:55,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:31:55,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-21 01:31:55,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:31:55,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdf4a8d54b5122738c69fd HTTP/1.1" 200 None 2026-03-21 01:31:55,625 - lava_callback - INFO - Completed processing callback for node 69bdf4a8d54b5122738c69fd INFO: 172.21.0.1:52942 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:31:59,508 - lava_callback - INFO - Received callback for node 69bdf4a6d54b5122738c69f6 INFO: 172.21.0.1:52956 - "POST /node/69bdf4a6d54b5122738c69f6 HTTP/1.0" 202 Accepted 2026-03-21 01:31:59,516 - lava_callback - INFO - Starting async job submit for node 69bdf4a6d54b5122738c69f6 2026-03-21 01:31:59,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:31:59,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf4a6d54b5122738c69f6 HTTP/1.1" 200 None 2026-03-21 01:31:59,562 - lava_callback - DEBUG - Processing node 69bdf4a6d54b5122738c69f6: baseline-nfs-arm64 2026-03-21 01:32:00,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:32:00,186 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:32:00,187 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69bdf4a6d54b5122738c69f6/log.txt.gz 2026-03-21 01:32:00,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:32:00,710 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:32:00,712 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69bdf4a6d54b5122738c69f6/lava-logs.yaml 2026-03-21 01:32:00,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:32:00,758 - lava_callback - INFO - Received callback for node 69bdf4add54b5122738c6a22 INFO: 172.21.0.1:37210 - "POST /node/69bdf4add54b5122738c6a22 HTTP/1.0" 202 Accepted 2026-03-21 01:32:00,765 - lava_callback - INFO - Starting async job submit for node 69bdf4add54b5122738c6a22 2026-03-21 01:32:00,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:32:00,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf4add54b5122738c6a22 HTTP/1.1" 200 None 2026-03-21 01:32:00,816 - lava_callback - DEBUG - Processing node 69bdf4add54b5122738c6a22: baseline-nfs-arm64 2026-03-21 01:32:00,930 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:32:00,987 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69bdf4a6d54b5122738c69f6/lava_callback.json.gz 2026-03-21 01:32:01,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:32:01,130 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf4a6d54b5122738c69f6 HTTP/1.1" 200 None 2026-03-21 01:32:01,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:32:01,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-21 01:32:01,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:32:01,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:32:01,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdf4a6d54b5122738c69f6 HTTP/1.1" 200 None 2026-03-21 01:32:01,306 - lava_callback - INFO - Completed processing callback for node 69bdf4a6d54b5122738c69f6 2026-03-21 01:32:01,508 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:32:01,510 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69bdf4add54b5122738c6a22/log.txt.gz 2026-03-21 01:32:01,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:32:01,992 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:32:01,994 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69bdf4add54b5122738c6a22/lava-logs.yaml 2026-03-21 01:32:02,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:32:02,177 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:32:02,178 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69bdf4add54b5122738c6a22/lava_callback.json.gz 2026-03-21 01:32:02,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:32:02,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf4add54b5122738c6a22 HTTP/1.1" 200 None 2026-03-21 01:32:02,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:32:02,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-21 01:32:02,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:32:02,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdf4add54b5122738c6a22 HTTP/1.1" 200 None 2026-03-21 01:32:02,269 - lava_callback - INFO - Completed processing callback for node 69bdf4add54b5122738c6a22 2026-03-21 01:32:02,775 - lava_callback - INFO - Received callback for node 69bdf494d54b5122738c69cd INFO: 172.21.0.1:37220 - "POST /node/69bdf494d54b5122738c69cd HTTP/1.0" 202 Accepted 2026-03-21 01:32:02,782 - lava_callback - INFO - Starting async job submit for node 69bdf494d54b5122738c69cd 2026-03-21 01:32:02,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:32:02,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf494d54b5122738c69cd HTTP/1.1" 200 None 2026-03-21 01:32:02,819 - lava_callback - DEBUG - Processing node 69bdf494d54b5122738c69cd: baseline-arm64 2026-03-21 01:32:03,339 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:32:03,495 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:32:03,497 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdf494d54b5122738c69cd/log.txt.gz 2026-03-21 01:32:03,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:32:03,939 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:32:03,941 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdf494d54b5122738c69cd/lava-logs.yaml 2026-03-21 01:32:03,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:32:04,108 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:32:04,110 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdf494d54b5122738c69cd/lava_callback.json.gz 2026-03-21 01:32:04,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:32:04,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf494d54b5122738c69cd HTTP/1.1" 200 None 2026-03-21 01:32:04,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:32:04,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-21 01:32:04,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:32:04,220 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdf494d54b5122738c69cd HTTP/1.1" 200 None 2026-03-21 01:32:04,263 - lava_callback - INFO - Completed processing callback for node 69bdf494d54b5122738c69cd INFO: 172.21.0.1:37236 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:32:06,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:32:06,204 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.21.0.1:37240 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:32:08,055 - lava_callback - INFO - Received callback for node 69bdf49fd54b5122738c69e6 INFO: 172.21.0.1:37254 - "POST /node/69bdf49fd54b5122738c69e6 HTTP/1.0" 202 Accepted 2026-03-21 01:32:08,062 - lava_callback - INFO - Starting async job submit for node 69bdf49fd54b5122738c69e6 2026-03-21 01:32:08,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:32:08,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf49fd54b5122738c69e6 HTTP/1.1" 200 None 2026-03-21 01:32:08,100 - lava_callback - DEBUG - Processing node 69bdf49fd54b5122738c69e6: baseline-nfs-arm64 2026-03-21 01:32:08,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:32:08,525 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:32:08,527 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69bdf49fd54b5122738c69e6/log.txt.gz 2026-03-21 01:32:08,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:32:08,904 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:32:08,906 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69bdf49fd54b5122738c69e6/lava-logs.yaml 2026-03-21 01:32:08,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:32:09,051 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:32:09,052 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69bdf49fd54b5122738c69e6/lava_callback.json.gz 2026-03-21 01:32:09,076 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:32:09,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf49fd54b5122738c69e6 HTTP/1.1" 200 None 2026-03-21 01:32:09,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:32:09,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-21 01:32:09,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:32:09,145 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdf49fd54b5122738c69e6 HTTP/1.1" 200 None 2026-03-21 01:32:09,150 - lava_callback - INFO - Completed processing callback for node 69bdf49fd54b5122738c69e6 2026-03-21 01:32:10,134 - lava_callback - INFO - Received callback for node 69bdf4c2d54b5122738c6a5e INFO: 172.21.0.1:56112 - "POST /node/69bdf4c2d54b5122738c6a5e HTTP/1.0" 202 Accepted 2026-03-21 01:32:10,140 - lava_callback - INFO - Starting async job submit for node 69bdf4c2d54b5122738c6a5e 2026-03-21 01:32:10,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:32:10,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf4c2d54b5122738c6a5e HTTP/1.1" 200 None 2026-03-21 01:32:10,178 - lava_callback - DEBUG - Processing node 69bdf4c2d54b5122738c6a5e: baseline-arm64-broonie 2026-03-21 01:32:10,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:32:10,555 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:32:10,557 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69bdf4c2d54b5122738c6a5e/log.txt.gz 2026-03-21 01:32:10,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:32:10,878 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:32:10,880 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69bdf4c2d54b5122738c6a5e/lava-logs.yaml 2026-03-21 01:32:10,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:32:11,010 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:32:11,011 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69bdf4c2d54b5122738c6a5e/lava_callback.json.gz 2026-03-21 01:32:11,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:32:11,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf4c2d54b5122738c6a5e HTTP/1.1" 200 None 2026-03-21 01:32:11,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:32:11,063 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-21 01:32:11,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:32:11,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdf4c2d54b5122738c6a5e HTTP/1.1" 200 None 2026-03-21 01:32:11,104 - lava_callback - INFO - Completed processing callback for node 69bdf4c2d54b5122738c6a5e 2026-03-21 01:32:11,961 - lava_callback - INFO - Received callback for node 69bdf50dd54b5122738c6b26 INFO: 172.21.0.1:56124 - "POST /node/69bdf50dd54b5122738c6b26 HTTP/1.0" 202 Accepted 2026-03-21 01:32:11,967 - lava_callback - INFO - Starting async job submit for node 69bdf50dd54b5122738c6b26 2026-03-21 01:32:11,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:32:11,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf50dd54b5122738c6b26 HTTP/1.1" 200 None 2026-03-21 01:32:11,984 - lava_callback - DEBUG - Processing node 69bdf50dd54b5122738c6b26: baseline-nfs-arm64 2026-03-21 01:32:11,985 - lava_callback - WARNING - Failed to upload log 2026-03-21 01:32:11,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:32:12,094 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:32:12,096 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69bdf50dd54b5122738c6b26/lava-logs.yaml 2026-03-21 01:32:12,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:32:12,211 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:32:12,213 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69bdf50dd54b5122738c6b26/lava_callback.json.gz 2026-03-21 01:32:12,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:32:12,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf50dd54b5122738c6b26 HTTP/1.1" 200 None 2026-03-21 01:32:12,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:32:12,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-21 01:32:12,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:32:12,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdf50dd54b5122738c6b26 HTTP/1.1" 200 None 2026-03-21 01:32:12,272 - lava_callback - INFO - Completed processing callback for node 69bdf50dd54b5122738c6b26 2026-03-21 01:32:12,490 - lava_callback - INFO - Received callback for node 69bdf4d0d54b5122738c6a9c INFO: 172.21.0.1:56126 - "POST /node/69bdf4d0d54b5122738c6a9c HTTP/1.0" 202 Accepted 2026-03-21 01:32:12,497 - lava_callback - INFO - Starting async job submit for node 69bdf4d0d54b5122738c6a9c 2026-03-21 01:32:12,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:32:12,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf4d0d54b5122738c6a9c HTTP/1.1" 200 None 2026-03-21 01:32:12,538 - lava_callback - DEBUG - Processing node 69bdf4d0d54b5122738c6a9c: baseline-arm64-pengutronix 2026-03-21 01:32:12,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:32:12,857 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:32:12,858 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69bdf4d0d54b5122738c6a9c/log.txt.gz 2026-03-21 01:32:12,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:32:13,129 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:32:13,130 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69bdf4d0d54b5122738c6a9c/lava-logs.yaml 2026-03-21 01:32:13,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:32:13,263 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:32:13,265 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69bdf4d0d54b5122738c6a9c/lava_callback.json.gz 2026-03-21 01:32:13,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:32:13,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf4d0d54b5122738c6a9c HTTP/1.1" 200 None 2026-03-21 01:32:13,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:32:13,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-21 01:32:13,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:32:13,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdf4d0d54b5122738c6a9c HTTP/1.1" 200 None 2026-03-21 01:32:13,347 - lava_callback - INFO - Completed processing callback for node 69bdf4d0d54b5122738c6a9c INFO: 172.21.0.1:56134 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56148 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:32:21,949 - lava_callback - INFO - Received callback for node 69bdf4d2d54b5122738c6aa5 INFO: 172.21.0.1:59064 - "POST /node/69bdf4d2d54b5122738c6aa5 HTTP/1.0" 202 Accepted 2026-03-21 01:32:21,957 - lava_callback - INFO - Starting async job submit for node 69bdf4d2d54b5122738c6aa5 2026-03-21 01:32:21,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:32:21,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf4d2d54b5122738c6aa5 HTTP/1.1" 200 None 2026-03-21 01:32:21,997 - lava_callback - DEBUG - Processing node 69bdf4d2d54b5122738c6aa5: baseline-arm64-pengutronix 2026-03-21 01:32:22,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:32:22,307 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:32:22,309 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69bdf4d2d54b5122738c6aa5/log.txt.gz 2026-03-21 01:32:22,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:32:22,607 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:32:22,609 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69bdf4d2d54b5122738c6aa5/lava-logs.yaml 2026-03-21 01:32:22,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:32:22,752 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:32:22,754 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69bdf4d2d54b5122738c6aa5/lava_callback.json.gz 2026-03-21 01:32:22,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:32:22,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf4d2d54b5122738c6aa5 HTTP/1.1" 200 None 2026-03-21 01:32:22,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:32:22,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-21 01:32:22,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:32:22,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdf4d2d54b5122738c6aa5 HTTP/1.1" 200 None 2026-03-21 01:32:22,842 - lava_callback - INFO - Completed processing callback for node 69bdf4d2d54b5122738c6aa5 2026-03-21 01:32:25,656 - lava_callback - INFO - Received callback for node 69bdf4d3d54b5122738c6aaa INFO: 172.21.0.1:59080 - "POST /node/69bdf4d3d54b5122738c6aaa HTTP/1.0" 202 Accepted 2026-03-21 01:32:25,662 - lava_callback - INFO - Starting async job submit for node 69bdf4d3d54b5122738c6aaa 2026-03-21 01:32:25,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:32:25,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf4d3d54b5122738c6aaa HTTP/1.1" 200 None 2026-03-21 01:32:25,696 - lava_callback - DEBUG - Processing node 69bdf4d3d54b5122738c6aaa: baseline-arm64-pengutronix 2026-03-21 01:32:25,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:32:26,034 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:32:26,036 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69bdf4d3d54b5122738c6aaa/log.txt.gz 2026-03-21 01:32:26,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:32:26,323 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:32:26,325 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69bdf4d3d54b5122738c6aaa/lava-logs.yaml 2026-03-21 01:32:26,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:32:26,502 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:32:26,504 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69bdf4d3d54b5122738c6aaa/lava_callback.json.gz 2026-03-21 01:32:26,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:59084 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:32:26,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf4d3d54b5122738c6aaa HTTP/1.1" 200 None 2026-03-21 01:32:26,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:32:26,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-21 01:32:26,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:32:26,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdf4d3d54b5122738c6aaa HTTP/1.1" 200 None 2026-03-21 01:32:26,613 - lava_callback - INFO - Completed processing callback for node 69bdf4d3d54b5122738c6aaa 2026-03-21 01:32:36,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:32:36,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.21.0.1:52960 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:32:40,266 - lava_callback - INFO - Received callback for node 69bdf4c0d54b5122738c6a58 INFO: 172.21.0.1:42548 - "POST /node/69bdf4c0d54b5122738c6a58 HTTP/1.0" 202 Accepted 2026-03-21 01:32:40,272 - lava_callback - INFO - Starting async job submit for node 69bdf4c0d54b5122738c6a58 2026-03-21 01:32:40,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:32:40,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf4c0d54b5122738c6a58 HTTP/1.1" 200 None 2026-03-21 01:32:40,309 - lava_callback - DEBUG - Processing node 69bdf4c0d54b5122738c6a58: baseline-arm64-broonie 2026-03-21 01:32:40,567 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:32:40,692 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:32:40,694 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69bdf4c0d54b5122738c6a58/log.txt.gz 2026-03-21 01:32:40,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:32:41,059 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:32:41,061 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69bdf4c0d54b5122738c6a58/lava-logs.yaml 2026-03-21 01:32:41,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:32:41,201 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:32:41,203 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69bdf4c0d54b5122738c6a58/lava_callback.json.gz 2026-03-21 01:32:41,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:32:41,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf4c0d54b5122738c6a58 HTTP/1.1" 200 None 2026-03-21 01:32:41,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:32:41,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-21 01:32:41,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:32:41,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdf4c0d54b5122738c6a58 HTTP/1.1" 200 None 2026-03-21 01:32:41,282 - lava_callback - INFO - Completed processing callback for node 69bdf4c0d54b5122738c6a58 2026-03-21 01:32:41,693 - lava_callback - INFO - Received callback for node 69bdf4c4d54b5122738c6a65 INFO: 172.21.0.1:42556 - "POST /node/69bdf4c4d54b5122738c6a65 HTTP/1.0" 202 Accepted 2026-03-21 01:32:41,700 - lava_callback - INFO - Starting async job submit for node 69bdf4c4d54b5122738c6a65 2026-03-21 01:32:41,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:32:41,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf4c4d54b5122738c6a65 HTTP/1.1" 200 None 2026-03-21 01:32:41,744 - lava_callback - DEBUG - Processing node 69bdf4c4d54b5122738c6a65: baseline-arm64-broonie 2026-03-21 01:32:42,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:32:42,157 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:32:42,158 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69bdf4c4d54b5122738c6a65/log.txt.gz 2026-03-21 01:32:42,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:32:42,513 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:32:42,514 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69bdf4c4d54b5122738c6a65/lava-logs.yaml 2026-03-21 01:32:42,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:32:42,654 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:32:42,656 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69bdf4c4d54b5122738c6a65/lava_callback.json.gz 2026-03-21 01:32:42,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:32:42,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf4c4d54b5122738c6a65 HTTP/1.1" 200 None 2026-03-21 01:32:42,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:32:42,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-21 01:32:42,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:32:42,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdf4c4d54b5122738c6a65 HTTP/1.1" 200 None 2026-03-21 01:32:42,734 - lava_callback - INFO - Completed processing callback for node 69bdf4c4d54b5122738c6a65 2026-03-21 01:32:43,155 - lava_callback - INFO - Received callback for node 69bdf4c6d54b5122738c6a68 INFO: 172.21.0.1:42564 - "POST /node/69bdf4c6d54b5122738c6a68 HTTP/1.0" 202 Accepted 2026-03-21 01:32:43,165 - lava_callback - INFO - Starting async job submit for node 69bdf4c6d54b5122738c6a68 2026-03-21 01:32:43,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:32:43,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf4c6d54b5122738c6a68 HTTP/1.1" 200 None 2026-03-21 01:32:43,227 - lava_callback - DEBUG - Processing node 69bdf4c6d54b5122738c6a68: baseline-arm64-broonie 2026-03-21 01:32:43,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:32:43,598 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:32:43,599 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69bdf4c6d54b5122738c6a68/log.txt.gz 2026-03-21 01:32:43,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:32:43,961 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:32:43,963 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69bdf4c6d54b5122738c6a68/lava-logs.yaml 2026-03-21 01:32:43,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:32:44,105 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:32:44,106 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69bdf4c6d54b5122738c6a68/lava_callback.json.gz 2026-03-21 01:32:44,129 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:32:44,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf4c6d54b5122738c6a68 HTTP/1.1" 200 None 2026-03-21 01:32:44,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:32:44,158 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-21 01:32:44,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:32:44,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdf4c6d54b5122738c6a68 HTTP/1.1" 200 None 2026-03-21 01:32:44,199 - lava_callback - INFO - Completed processing callback for node 69bdf4c6d54b5122738c6a68 INFO: 172.21.0.1:42572 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:32:50,179 - lava_callback - INFO - Received callback for node 69bdf4c7d54b5122738c6a6b INFO: 172.21.0.1:55776 - "POST /node/69bdf4c7d54b5122738c6a6b HTTP/1.0" 202 Accepted 2026-03-21 01:32:50,185 - lava_callback - INFO - Starting async job submit for node 69bdf4c7d54b5122738c6a6b 2026-03-21 01:32:50,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:32:50,221 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf4c7d54b5122738c6a6b HTTP/1.1" 200 None 2026-03-21 01:32:50,222 - lava_callback - DEBUG - Processing node 69bdf4c7d54b5122738c6a6b: baseline-arm64-broonie 2026-03-21 01:32:50,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:32:50,611 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:32:50,613 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69bdf4c7d54b5122738c6a6b/log.txt.gz 2026-03-21 01:32:50,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:32:50,932 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:32:50,933 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69bdf4c7d54b5122738c6a6b/lava-logs.yaml 2026-03-21 01:32:50,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:32:51,080 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:32:51,082 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69bdf4c7d54b5122738c6a6b/lava_callback.json.gz 2026-03-21 01:32:51,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:32:51,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf4c7d54b5122738c6a6b HTTP/1.1" 200 None 2026-03-21 01:32:51,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:32:51,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-21 01:32:51,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:32:51,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdf4c7d54b5122738c6a6b HTTP/1.1" 200 None 2026-03-21 01:32:51,165 - lava_callback - INFO - Completed processing callback for node 69bdf4c7d54b5122738c6a6b INFO: 172.21.0.1:55788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38320 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:33:05,015 - lava_callback - INFO - Received callback for node 69bdf48fd54b5122738c69c1 INFO: 172.21.0.1:38324 - "POST /node/69bdf48fd54b5122738c69c1 HTTP/1.0" 202 Accepted 2026-03-21 01:33:05,021 - lava_callback - INFO - Starting async job submit for node 69bdf48fd54b5122738c69c1 2026-03-21 01:33:05,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:33:05,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf48fd54b5122738c69c1 HTTP/1.1" 200 None 2026-03-21 01:33:05,066 - lava_callback - DEBUG - Processing node 69bdf48fd54b5122738c69c1: baseline-arm64 2026-03-21 01:33:05,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:33:05,603 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:33:05,604 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdf48fd54b5122738c69c1/log.txt.gz 2026-03-21 01:33:05,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:33:06,048 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:33:06,050 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdf48fd54b5122738c69c1/lava-logs.yaml 2026-03-21 01:33:06,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:33:06,208 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:33:06,209 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdf48fd54b5122738c69c1/lava_callback.json.gz 2026-03-21 01:33:06,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:33:06,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:33:06,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf48fd54b5122738c69c1 HTTP/1.1" 200 None 2026-03-21 01:33:06,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:33:06,247 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-21 01:33:06,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-21 01:33:06,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:33:06,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdf48fd54b5122738c69c1 HTTP/1.1" 200 None 2026-03-21 01:33:06,299 - lava_callback - INFO - Completed processing callback for node 69bdf48fd54b5122738c69c1 INFO: 172.21.0.1:38326 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:33:10,520 - lava_callback - INFO - Received callback for node 69bdf4bbd54b5122738c6a48 INFO: 172.21.0.1:33430 - "POST /node/69bdf4bbd54b5122738c6a48 HTTP/1.0" 202 Accepted 2026-03-21 01:33:10,526 - lava_callback - INFO - Starting async job submit for node 69bdf4bbd54b5122738c6a48 2026-03-21 01:33:10,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:33:10,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf4bbd54b5122738c6a48 HTTP/1.1" 200 None 2026-03-21 01:33:10,565 - lava_callback - DEBUG - Processing node 69bdf4bbd54b5122738c6a48: baseline-arm64-broonie 2026-03-21 01:33:10,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:33:10,904 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:33:10,905 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69bdf4bbd54b5122738c6a48/log.txt.gz 2026-03-21 01:33:11,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:33:11,260 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:33:11,262 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69bdf4bbd54b5122738c6a48/lava-logs.yaml 2026-03-21 01:33:11,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:33:11,411 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:33:11,412 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69bdf4bbd54b5122738c6a48/lava_callback.json.gz 2026-03-21 01:33:11,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:33:11,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf4bbd54b5122738c6a48 HTTP/1.1" 200 None 2026-03-21 01:33:11,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:33:11,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-21 01:33:11,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:33:11,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdf4bbd54b5122738c6a48 HTTP/1.1" 200 None 2026-03-21 01:33:11,503 - lava_callback - INFO - Completed processing callback for node 69bdf4bbd54b5122738c6a48 INFO: 172.21.0.1:33442 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43578 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:33:36,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:33:36,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:55864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34698 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:33:52,962 - lava_callback - INFO - Received callback for node 69bb526ca1e2bfb1b33471f6 INFO: 172.21.0.1:33622 - "POST /node/69bb526ca1e2bfb1b33471f6 HTTP/1.0" 202 Accepted 2026-03-21 01:33:52,968 - lava_callback - INFO - Starting async job submit for node 69bb526ca1e2bfb1b33471f6 2026-03-21 01:33:52,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:33:52,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb526ca1e2bfb1b33471f6 HTTP/1.1" 200 None 2026-03-21 01:33:52,988 - lava_callback - DEBUG - Processing node 69bb526ca1e2bfb1b33471f6: baseline-arm-clabbe 2026-03-21 01:33:52,988 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb526ca1e2bfb1b33471f6 2026-03-21 01:33:52,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:33:53,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb526ca1e2bfb1b33471f6 HTTP/1.1" 200 None 2026-03-21 01:33:53,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:33:53,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-21 01:33:53,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:33:53,044 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb526ca1e2bfb1b33471f6 HTTP/1.1" 200 None 2026-03-21 01:33:53,046 - lava_callback - ERROR - Error processing callback for node 69bb526ca1e2bfb1b33471f6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41762 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:34:12,978 - lava_callback - INFO - Received callback for node 69bb5291a1e2bfb1b3347215 INFO: 172.21.0.1:43262 - "POST /node/69bb5291a1e2bfb1b3347215 HTTP/1.0" 202 Accepted 2026-03-21 01:34:12,983 - lava_callback - INFO - Starting async job submit for node 69bb5291a1e2bfb1b3347215 2026-03-21 01:34:12,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:34:12,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb5291a1e2bfb1b3347215 HTTP/1.1" 200 None 2026-03-21 01:34:13,000 - lava_callback - DEBUG - Processing node 69bb5291a1e2bfb1b3347215: baseline-arm-clabbe 2026-03-21 01:34:13,000 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb5291a1e2bfb1b3347215 2026-03-21 01:34:13,001 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:34:13,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb5291a1e2bfb1b3347215 HTTP/1.1" 200 None 2026-03-21 01:34:13,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:34:13,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-21 01:34:13,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:34:13,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb5291a1e2bfb1b3347215 HTTP/1.1" 200 None 2026-03-21 01:34:13,057 - lava_callback - ERROR - Error processing callback for node 69bb5291a1e2bfb1b3347215: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43278 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60120 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:34:26,634 - lava_callback - INFO - Received callback for node 69bdf4a0d54b5122738c69e9 INFO: 172.21.0.1:60136 - "POST /node/69bdf4a0d54b5122738c69e9 HTTP/1.0" 202 Accepted 2026-03-21 01:34:26,640 - lava_callback - INFO - Starting async job submit for node 69bdf4a0d54b5122738c69e9 2026-03-21 01:34:26,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:34:26,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf4a0d54b5122738c69e9 HTTP/1.1" 200 None 2026-03-21 01:34:26,674 - lava_callback - DEBUG - Processing node 69bdf4a0d54b5122738c69e9: baseline-nfs-arm64 2026-03-21 01:34:27,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:34:27,248 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:34:27,249 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69bdf4a0d54b5122738c69e9/log.txt.gz 2026-03-21 01:34:27,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:34:27,709 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:34:27,711 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69bdf4a0d54b5122738c69e9/lava-logs.yaml 2026-03-21 01:34:27,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:34:27,928 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:34:27,930 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69bdf4a0d54b5122738c69e9/lava_callback.json.gz 2026-03-21 01:34:27,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:34:27,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf4a0d54b5122738c69e9 HTTP/1.1" 200 None 2026-03-21 01:34:27,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:34:27,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-21 01:34:27,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:34:28,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdf4a0d54b5122738c69e9 HTTP/1.1" 200 None 2026-03-21 01:34:28,011 - lava_callback - INFO - Completed processing callback for node 69bdf4a0d54b5122738c69e9 2026-03-21 01:34:30,237 - lava_callback - INFO - Received callback for node 69bdf4c1d54b5122738c6a5b INFO: 172.21.0.1:37986 - "POST /node/69bdf4c1d54b5122738c6a5b HTTP/1.0" 202 Accepted 2026-03-21 01:34:30,243 - lava_callback - INFO - Starting async job submit for node 69bdf4c1d54b5122738c6a5b 2026-03-21 01:34:30,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:34:30,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf4c1d54b5122738c6a5b HTTP/1.1" 200 None 2026-03-21 01:34:30,280 - lava_callback - DEBUG - Processing node 69bdf4c1d54b5122738c6a5b: baseline-arm64-broonie 2026-03-21 01:34:30,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:34:30,704 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:34:30,706 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69bdf4c1d54b5122738c6a5b/log.txt.gz 2026-03-21 01:34:30,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:34:31,062 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:34:31,064 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69bdf4c1d54b5122738c6a5b/lava-logs.yaml 2026-03-21 01:34:31,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:34:31,228 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:34:31,230 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69bdf4c1d54b5122738c6a5b/lava_callback.json.gz 2026-03-21 01:34:31,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:34:31,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf4c1d54b5122738c6a5b HTTP/1.1" 200 None 2026-03-21 01:34:31,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:34:31,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-21 01:34:31,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:34:31,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdf4c1d54b5122738c6a5b HTTP/1.1" 200 None 2026-03-21 01:34:31,323 - lava_callback - INFO - Completed processing callback for node 69bdf4c1d54b5122738c6a5b 2026-03-21 01:34:36,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:34:36,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:37990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58212 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:34:52,987 - lava_callback - INFO - Received callback for node 69bb52b6a1e2bfb1b334721c INFO: 172.21.0.1:50006 - "POST /node/69bb52b6a1e2bfb1b334721c HTTP/1.0" 202 Accepted 2026-03-21 01:34:52,992 - lava_callback - INFO - Starting async job submit for node 69bb52b6a1e2bfb1b334721c 2026-03-21 01:34:52,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:34:53,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb52b6a1e2bfb1b334721c HTTP/1.1" 200 None 2026-03-21 01:34:53,021 - lava_callback - DEBUG - Processing node 69bb52b6a1e2bfb1b334721c: baseline-arm-clabbe 2026-03-21 01:34:53,021 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb52b6a1e2bfb1b334721c 2026-03-21 01:34:53,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:34:53,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb52b6a1e2bfb1b334721c HTTP/1.1" 200 None 2026-03-21 01:34:53,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:34:53,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-21 01:34:53,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:34:53,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb52b6a1e2bfb1b334721c HTTP/1.1" 200 None 2026-03-21 01:34:53,076 - lava_callback - ERROR - Error processing callback for node 69bb52b6a1e2bfb1b334721c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56822 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58398 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:35:32,975 - lava_callback - INFO - Received callback for node 69bb52daa1e2bfb1b3347227 INFO: 172.21.0.1:51180 - "POST /node/69bb52daa1e2bfb1b3347227 HTTP/1.0" 202 Accepted 2026-03-21 01:35:32,981 - lava_callback - INFO - Starting async job submit for node 69bb52daa1e2bfb1b3347227 2026-03-21 01:35:32,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:35:32,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb52daa1e2bfb1b3347227 HTTP/1.1" 200 None 2026-03-21 01:35:32,997 - lava_callback - DEBUG - Processing node 69bb52daa1e2bfb1b3347227: baseline-arm-clabbe 2026-03-21 01:35:32,997 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb52daa1e2bfb1b3347227 2026-03-21 01:35:32,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:35:33,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb52daa1e2bfb1b3347227 HTTP/1.1" 200 None 2026-03-21 01:35:33,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:35:33,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-21 01:35:33,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:35:33,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb52daa1e2bfb1b3347227 HTTP/1.1" 200 None 2026-03-21 01:35:33,098 - lava_callback - ERROR - Error processing callback for node 69bb52daa1e2bfb1b3347227: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51186 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:35:44,548 - lava_callback - INFO - Received callback for node 69bdf5aad54b5122738c6bc6 INFO: 172.21.0.1:50748 - "POST /node/69bdf5aad54b5122738c6bc6 HTTP/1.0" 202 Accepted 2026-03-21 01:35:44,555 - lava_callback - INFO - Starting async job submit for node 69bdf5aad54b5122738c6bc6 2026-03-21 01:35:44,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:35:44,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf5aad54b5122738c6bc6 HTTP/1.1" 200 None 2026-03-21 01:35:44,599 - lava_callback - DEBUG - Processing node 69bdf5aad54b5122738c6bc6: baseline-arm64 2026-03-21 01:35:44,738 - lava_callback - WARNING - Failed to upload log 2026-03-21 01:35:44,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:35:45,018 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:35:45,020 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdf5aad54b5122738c6bc6/lava-logs.yaml 2026-03-21 01:35:45,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:35:45,169 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:35:45,170 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdf5aad54b5122738c6bc6/lava_callback.json.gz 2026-03-21 01:35:45,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:35:45,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf5aad54b5122738c6bc6 HTTP/1.1" 200 None 2026-03-21 01:35:45,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:35:45,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-21 01:35:45,239 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:35:45,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdf5aad54b5122738c6bc6 HTTP/1.1" 200 None 2026-03-21 01:35:45,329 - lava_callback - INFO - Completed processing callback for node 69bdf5aad54b5122738c6bc6 INFO: 172.21.0.1:50754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39128 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:36:06,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:36:06,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:39136 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:36:13,087 - lava_callback - INFO - Received callback for node 69bb5300a1e2bfb1b3347230 INFO: 172.21.0.1:54726 - "POST /node/69bb5300a1e2bfb1b3347230 HTTP/1.0" 202 Accepted 2026-03-21 01:36:13,092 - lava_callback - INFO - Starting async job submit for node 69bb5300a1e2bfb1b3347230 2026-03-21 01:36:13,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:36:13,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb5300a1e2bfb1b3347230 HTTP/1.1" 200 None 2026-03-21 01:36:13,118 - lava_callback - DEBUG - Processing node 69bb5300a1e2bfb1b3347230: baseline-arm-clabbe 2026-03-21 01:36:13,118 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb5300a1e2bfb1b3347230 2026-03-21 01:36:13,119 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:36:13,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb5300a1e2bfb1b3347230 HTTP/1.1" 200 None 2026-03-21 01:36:13,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:36:13,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-21 01:36:13,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:36:13,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb5300a1e2bfb1b3347230 HTTP/1.1" 200 None 2026-03-21 01:36:13,179 - lava_callback - ERROR - Error processing callback for node 69bb5300a1e2bfb1b3347230: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54740 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40856 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:36:46,322 - lava_callback - INFO - Received callback for node 69bdf4d5d54b5122738c6ab4 INFO: 172.21.0.1:46992 - "POST /node/69bdf4d5d54b5122738c6ab4 HTTP/1.0" 202 Accepted 2026-03-21 01:36:46,331 - lava_callback - INFO - Starting async job submit for node 69bdf4d5d54b5122738c6ab4 2026-03-21 01:36:46,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:36:46,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf4d5d54b5122738c6ab4 HTTP/1.1" 200 None 2026-03-21 01:36:46,400 - lava_callback - DEBUG - Processing node 69bdf4d5d54b5122738c6ab4: baseline-arm64-foundriesio INFO: 172.21.0.1:47000 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:36:47,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:36:47,721 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:36:47,723 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69bdf4d5d54b5122738c6ab4/log.txt.gz 2026-03-21 01:36:48,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:36:48,694 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:36:48,697 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69bdf4d5d54b5122738c6ab4/lava-logs.yaml 2026-03-21 01:36:48,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:36:48,928 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:36:48,930 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69bdf4d5d54b5122738c6ab4/lava_callback.json.gz DEBUG: Setup failed for 69bdf4d5d54b5122738c6ab4. Transitting job node result: incomplete -> fail 2026-03-21 01:36:49,045 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:36:49,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf4d5d54b5122738c6ab4 HTTP/1.1" 200 None 2026-03-21 01:36:49,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:36:49,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-21 01:36:49,076 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:36:49,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdf4d5d54b5122738c6ab4 HTTP/1.1" 200 None 2026-03-21 01:36:49,215 - lava_callback - INFO - Completed processing callback for node 69bdf4d5d54b5122738c6ab4 2026-03-21 01:36:53,041 - lava_callback - INFO - Received callback for node 69bb5324a1e2bfb1b334723b INFO: 172.21.0.1:51822 - "POST /node/69bb5324a1e2bfb1b334723b HTTP/1.0" 202 Accepted 2026-03-21 01:36:53,047 - lava_callback - INFO - Starting async job submit for node 69bb5324a1e2bfb1b334723b 2026-03-21 01:36:53,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:36:53,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb5324a1e2bfb1b334723b HTTP/1.1" 200 None 2026-03-21 01:36:53,063 - lava_callback - DEBUG - Processing node 69bb5324a1e2bfb1b334723b: baseline-arm-clabbe 2026-03-21 01:36:53,063 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb5324a1e2bfb1b334723b 2026-03-21 01:36:53,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:36:53,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb5324a1e2bfb1b334723b HTTP/1.1" 200 None 2026-03-21 01:36:53,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:36:53,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-21 01:36:53,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:36:53,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb5324a1e2bfb1b334723b HTTP/1.1" 200 None 2026-03-21 01:36:53,131 - lava_callback - ERROR - Error processing callback for node 69bb5324a1e2bfb1b334723b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-21 01:36:54,693 - lava_callback - INFO - Received callback for node 69bdf5f9d54b5122738c6bdd INFO: 172.21.0.1:51838 - "POST /node/69bdf5f9d54b5122738c6bdd HTTP/1.0" 202 Accepted 2026-03-21 01:36:54,699 - lava_callback - INFO - Starting async job submit for node 69bdf5f9d54b5122738c6bdd 2026-03-21 01:36:54,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:36:54,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf5f9d54b5122738c6bdd HTTP/1.1" 200 None 2026-03-21 01:36:54,743 - lava_callback - DEBUG - Processing node 69bdf5f9d54b5122738c6bdd: baseline-arm64 2026-03-21 01:36:54,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:36:55,017 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:36:55,019 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdf5f9d54b5122738c6bdd/log.txt.gz 2026-03-21 01:36:55,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:36:55,286 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:36:55,288 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdf5f9d54b5122738c6bdd/lava-logs.yaml 2026-03-21 01:36:55,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:36:55,434 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:36:55,436 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdf5f9d54b5122738c6bdd/lava_callback.json.gz 2026-03-21 01:36:55,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:36:55,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf5f9d54b5122738c6bdd HTTP/1.1" 200 None 2026-03-21 01:36:55,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:36:55,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-21 01:36:55,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:36:55,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdf5f9d54b5122738c6bdd HTTP/1.1" 200 None 2026-03-21 01:36:55,597 - lava_callback - INFO - Completed processing callback for node 69bdf5f9d54b5122738c6bdd INFO: 172.21.0.1:51852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38776 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:37:06,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:37:06,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:38792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57598 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35522 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:37:29,876 - lava_callback - INFO - Received callback for node 69bdf49ed54b5122738c69e3 INFO: 172.21.0.1:32880 - "POST /node/69bdf49ed54b5122738c69e3 HTTP/1.0" 202 Accepted 2026-03-21 01:37:29,884 - lava_callback - INFO - Starting async job submit for node 69bdf49ed54b5122738c69e3 2026-03-21 01:37:29,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:37:29,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf49ed54b5122738c69e3 HTTP/1.1" 200 None 2026-03-21 01:37:29,934 - lava_callback - DEBUG - Processing node 69bdf49ed54b5122738c69e3: baseline-arm64 2026-03-21 01:37:30,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:37:30,420 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:37:30,421 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdf49ed54b5122738c69e3/log.txt.gz 2026-03-21 01:37:30,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:37:30,866 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:37:30,868 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdf49ed54b5122738c69e3/lava-logs.yaml 2026-03-21 01:37:30,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:37:31,033 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:37:31,034 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdf49ed54b5122738c69e3/lava_callback.json.gz DEBUG: Setup failed for 69bdf49ed54b5122738c69e3. Transitting job node result: incomplete -> fail 2026-03-21 01:37:31,107 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:37:31,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf49ed54b5122738c69e3 HTTP/1.1" 200 None 2026-03-21 01:37:31,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:37:31,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-21 01:37:31,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:37:31,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdf49ed54b5122738c69e3 HTTP/1.1" 200 None 2026-03-21 01:37:31,258 - lava_callback - INFO - Completed processing callback for node 69bdf49ed54b5122738c69e3 2026-03-21 01:37:33,091 - lava_callback - INFO - Received callback for node 69bb5348a1e2bfb1b3347242 INFO: 172.21.0.1:32886 - "POST /node/69bb5348a1e2bfb1b3347242 HTTP/1.0" 202 Accepted 2026-03-21 01:37:33,097 - lava_callback - INFO - Starting async job submit for node 69bb5348a1e2bfb1b3347242 2026-03-21 01:37:33,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:37:33,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb5348a1e2bfb1b3347242 HTTP/1.1" 200 None 2026-03-21 01:37:33,114 - lava_callback - DEBUG - Processing node 69bb5348a1e2bfb1b3347242: baseline-arm-clabbe 2026-03-21 01:37:33,114 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb5348a1e2bfb1b3347242 2026-03-21 01:37:33,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:37:33,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb5348a1e2bfb1b3347242 HTTP/1.1" 200 None 2026-03-21 01:37:33,129 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:37:33,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-21 01:37:33,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:37:33,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb5348a1e2bfb1b3347242 HTTP/1.1" 200 None 2026-03-21 01:37:33,175 - lava_callback - ERROR - Error processing callback for node 69bb5348a1e2bfb1b3347242: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-21 01:37:36,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:37:36,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:32890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51816 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:37:53,079 - lava_callback - INFO - Received callback for node 69bb536ca1e2bfb1b334724d INFO: 172.21.0.1:41560 - "POST /node/69bb536ca1e2bfb1b334724d HTTP/1.0" 202 Accepted 2026-03-21 01:37:53,085 - lava_callback - INFO - Starting async job submit for node 69bb536ca1e2bfb1b334724d 2026-03-21 01:37:53,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:37:53,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb536ca1e2bfb1b334724d HTTP/1.1" 200 None 2026-03-21 01:37:53,100 - lava_callback - DEBUG - Processing node 69bb536ca1e2bfb1b334724d: baseline-arm-clabbe 2026-03-21 01:37:53,100 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb536ca1e2bfb1b334724d 2026-03-21 01:37:53,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:37:53,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb536ca1e2bfb1b334724d HTTP/1.1" 200 None 2026-03-21 01:37:53,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:37:53,129 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-21 01:37:53,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:37:53,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb536ca1e2bfb1b334724d HTTP/1.1" 200 None 2026-03-21 01:37:53,163 - lava_callback - ERROR - Error processing callback for node 69bb536ca1e2bfb1b334724d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36440 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:38:14,610 - lava_callback - INFO - Received callback for node 69bdf649d54b5122738c6bf9 INFO: 172.21.0.1:42624 - "POST /node/69bdf649d54b5122738c6bf9 HTTP/1.0" 202 Accepted 2026-03-21 01:38:14,617 - lava_callback - INFO - Starting async job submit for node 69bdf649d54b5122738c6bf9 2026-03-21 01:38:14,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:38:14,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf649d54b5122738c6bf9 HTTP/1.1" 200 None 2026-03-21 01:38:14,653 - lava_callback - DEBUG - Processing node 69bdf649d54b5122738c6bf9: baseline-arm64 2026-03-21 01:38:15,076 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:42638 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-21 01:38:15,260 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:38:15,262 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdf649d54b5122738c6bf9/log.txt.gz 2026-03-21 01:38:15,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:38:15,711 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:38:15,713 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdf649d54b5122738c6bf9/lava-logs.yaml 2026-03-21 01:38:15,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:38:15,872 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:38:15,873 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdf649d54b5122738c6bf9/lava_callback.json.gz 2026-03-21 01:38:15,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:38:15,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf649d54b5122738c6bf9 HTTP/1.1" 200 None 2026-03-21 01:38:15,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:38:15,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-21 01:38:15,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:38:15,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdf649d54b5122738c6bf9 HTTP/1.1" 200 None 2026-03-21 01:38:15,972 - lava_callback - INFO - Completed processing callback for node 69bdf649d54b5122738c6bf9 INFO: 172.21.0.1:42646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58264 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:38:36,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:38:36,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:37486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55006 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:38:52,162 - lava_callback - INFO - Received callback for node 69bdf69dd54b5122738c6c16 INFO: 172.21.0.1:40054 - "POST /node/69bdf69dd54b5122738c6c16 HTTP/1.0" 202 Accepted 2026-03-21 01:38:52,168 - lava_callback - INFO - Starting async job submit for node 69bdf69dd54b5122738c6c16 2026-03-21 01:38:52,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:38:52,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf69dd54b5122738c6c16 HTTP/1.1" 200 None 2026-03-21 01:38:52,184 - lava_callback - DEBUG - Processing node 69bdf69dd54b5122738c6c16: baseline-nfs-arm64 2026-03-21 01:38:52,186 - lava_callback - WARNING - Failed to upload log 2026-03-21 01:38:52,187 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:38:52,308 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:38:52,309 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69bdf69dd54b5122738c6c16/lava-logs.yaml 2026-03-21 01:38:52,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:38:52,446 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:38:52,448 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69bdf69dd54b5122738c6c16/lava_callback.json.gz 2026-03-21 01:38:52,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:38:52,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf69dd54b5122738c6c16 HTTP/1.1" 200 None 2026-03-21 01:38:52,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:38:52,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-21 01:38:52,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:38:52,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdf69dd54b5122738c6c16 HTTP/1.1" 200 None 2026-03-21 01:38:52,505 - lava_callback - INFO - Completed processing callback for node 69bdf69dd54b5122738c6c16 INFO: 172.21.0.1:40058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57474 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:39:06,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:39:06,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:57484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60944 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43006 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:39:29,367 - lava_callback - INFO - Received callback for node 69bdf4cdd54b5122738c6a93 INFO: 172.21.0.1:43012 - "POST /node/69bdf4cdd54b5122738c6a93 HTTP/1.0" 202 Accepted 2026-03-21 01:39:29,373 - lava_callback - INFO - Starting async job submit for node 69bdf4cdd54b5122738c6a93 2026-03-21 01:39:29,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:39:29,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf4cdd54b5122738c6a93 HTTP/1.1" 200 None 2026-03-21 01:39:29,419 - lava_callback - DEBUG - Processing node 69bdf4cdd54b5122738c6a93: baseline-arm64-clabbe 2026-03-21 01:39:29,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:39:29,994 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:39:29,995 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69bdf4cdd54b5122738c6a93/log.txt.gz 2026-03-21 01:39:30,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:39:30,500 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:39:30,502 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69bdf4cdd54b5122738c6a93/lava-logs.yaml 2026-03-21 01:39:30,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:39:30,679 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:39:30,680 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69bdf4cdd54b5122738c6a93/lava_callback.json.gz 2026-03-21 01:39:30,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:39:30,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf4cdd54b5122738c6a93 HTTP/1.1" 200 None 2026-03-21 01:39:30,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:39:30,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-21 01:39:30,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:39:30,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdf4cdd54b5122738c6a93 HTTP/1.1" 200 None 2026-03-21 01:39:30,770 - lava_callback - INFO - Completed processing callback for node 69bdf4cdd54b5122738c6a93 2026-03-21 01:39:36,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:39:36,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:43298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58260 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35262 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:40:39,948 - lava_callback - INFO - Received callback for node 69bb540da1e2bfb1b3347289 INFO: 172.21.0.1:52974 - "POST /node/69bb540da1e2bfb1b3347289 HTTP/1.0" 202 Accepted 2026-03-21 01:40:39,954 - lava_callback - INFO - Starting async job submit for node 69bb540da1e2bfb1b3347289 2026-03-21 01:40:39,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:40:39,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb540da1e2bfb1b3347289 HTTP/1.1" 200 None 2026-03-21 01:40:39,969 - lava_callback - DEBUG - Processing node 69bb540da1e2bfb1b3347289: baseline-arm64-baylibre 2026-03-21 01:40:39,970 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb540da1e2bfb1b3347289 2026-03-21 01:40:39,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:40:39,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb540da1e2bfb1b3347289 HTTP/1.1" 200 None 2026-03-21 01:40:39,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:40:40,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-21 01:40:40,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:40:40,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb540da1e2bfb1b3347289 HTTP/1.1" 200 None 2026-03-21 01:40:40,026 - lava_callback - ERROR - Error processing callback for node 69bb540da1e2bfb1b3347289: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56868 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:40:58,070 - lava_callback - INFO - Received callback for node 69bdf497d54b5122738c69d7 INFO: 172.21.0.1:56870 - "POST /node/69bdf497d54b5122738c69d7 HTTP/1.0" 202 Accepted 2026-03-21 01:40:58,076 - lava_callback - INFO - Starting async job submit for node 69bdf497d54b5122738c69d7 2026-03-21 01:40:58,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:40:58,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf497d54b5122738c69d7 HTTP/1.1" 200 None 2026-03-21 01:40:58,116 - lava_callback - DEBUG - Processing node 69bdf497d54b5122738c69d7: baseline-arm64 2026-03-21 01:40:58,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:40:58,756 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:40:58,758 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdf497d54b5122738c69d7/log.txt.gz 2026-03-21 01:40:59,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:40:59,405 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:40:59,407 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdf497d54b5122738c69d7/lava-logs.yaml 2026-03-21 01:40:59,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:40:59,601 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:40:59,603 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdf497d54b5122738c69d7/lava_callback.json.gz 2026-03-21 01:40:59,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:40:59,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf497d54b5122738c69d7 HTTP/1.1" 200 None 2026-03-21 01:40:59,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:40:59,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-21 01:40:59,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:40:59,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdf497d54b5122738c69d7 HTTP/1.1" 200 None 2026-03-21 01:40:59,774 - lava_callback - INFO - Completed processing callback for node 69bdf497d54b5122738c69d7 INFO: 172.21.0.1:52058 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:41:06,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:41:06,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:52060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47642 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54528 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:41:31,495 - lava_callback - INFO - Received callback for node 69bdf32cd54b5122738c695c INFO: 172.21.0.1:55700 - "POST /node/69bdf32cd54b5122738c695c HTTP/1.0" 202 Accepted 2026-03-21 01:41:31,501 - lava_callback - INFO - Starting async job submit for node 69bdf32cd54b5122738c695c 2026-03-21 01:41:31,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:41:31,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf32cd54b5122738c695c HTTP/1.1" 200 None 2026-03-21 01:41:31,574 - lava_callback - DEBUG - Processing node 69bdf32cd54b5122738c695c: ltp-smoke 2026-03-21 01:41:32,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:41:32,174 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:41:32,176 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69bdf32cd54b5122738c695c/log.txt.gz 2026-03-21 01:41:32,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:41:32,672 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:41:32,675 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69bdf32cd54b5122738c695c/lava-logs.yaml 2026-03-21 01:41:32,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:41:32,835 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:41:32,837 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69bdf32cd54b5122738c695c/lava_callback.json.gz 2026-03-21 01:41:32,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:41:32,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf32cd54b5122738c695c HTTP/1.1" 200 None 2026-03-21 01:41:32,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:41:32,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde35fd54b5122738c3d99 HTTP/1.1" 200 None 2026-03-21 01:41:32,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:41:33,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdf32cd54b5122738c695c HTTP/1.1" 200 None 2026-03-21 01:41:33,167 - lava_callback - INFO - Completed processing callback for node 69bdf32cd54b5122738c695c 2026-03-21 01:41:36,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:41:36,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:55706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58932 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:41:59,969 - lava_callback - INFO - Received callback for node 69bb545ea1e2bfb1b3347294 INFO: 172.21.0.1:52840 - "POST /node/69bb545ea1e2bfb1b3347294 HTTP/1.0" 202 Accepted 2026-03-21 01:41:59,978 - lava_callback - INFO - Starting async job submit for node 69bb545ea1e2bfb1b3347294 2026-03-21 01:41:59,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:41:59,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb545ea1e2bfb1b3347294 HTTP/1.1" 200 None 2026-03-21 01:41:59,997 - lava_callback - DEBUG - Processing node 69bb545ea1e2bfb1b3347294: baseline-arm64-baylibre 2026-03-21 01:41:59,998 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb545ea1e2bfb1b3347294 2026-03-21 01:41:59,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:42:00,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb545ea1e2bfb1b3347294 HTTP/1.1" 200 None 2026-03-21 01:42:00,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:42:00,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-21 01:42:00,045 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:42:00,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb545ea1e2bfb1b3347294 HTTP/1.1" 200 None 2026-03-21 01:42:00,091 - lava_callback - ERROR - Error processing callback for node 69bb545ea1e2bfb1b3347294: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52868 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:42:07,856 - lava_callback - INFO - Received callback for node 69bdf32dd54b5122738c695f INFO: 172.21.0.1:52870 - "POST /node/69bdf32dd54b5122738c695f HTTP/1.0" 202 Accepted 2026-03-21 01:42:07,863 - lava_callback - INFO - Starting async job submit for node 69bdf32dd54b5122738c695f 2026-03-21 01:42:07,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:42:07,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf32dd54b5122738c695f HTTP/1.1" 200 None 2026-03-21 01:42:07,930 - lava_callback - DEBUG - Processing node 69bdf32dd54b5122738c695f: baseline-arm 2026-03-21 01:42:08,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:42:08,627 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:42:08,628 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69bdf32dd54b5122738c695f/log.txt.gz 2026-03-21 01:42:09,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:42:09,272 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:42:09,274 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69bdf32dd54b5122738c695f/lava-logs.yaml 2026-03-21 01:42:09,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:42:09,464 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:42:09,466 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69bdf32dd54b5122738c695f/lava_callback.json.gz DEBUG: Setup failed for 69bdf32dd54b5122738c695f. Transitting job node result: incomplete -> fail 2026-03-21 01:42:09,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:42:09,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf32dd54b5122738c695f HTTP/1.1" 200 None 2026-03-21 01:42:09,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:42:09,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-21 01:42:09,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:42:09,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdf32dd54b5122738c695f HTTP/1.1" 200 None 2026-03-21 01:42:09,740 - lava_callback - INFO - Completed processing callback for node 69bdf32dd54b5122738c695f 2026-03-21 01:42:13,061 - lava_callback - INFO - Received callback for node 69bdf731d54b5122738c6c3d INFO: 172.21.0.1:59264 - "POST /node/69bdf731d54b5122738c6c3d HTTP/1.0" 202 Accepted 2026-03-21 01:42:13,067 - lava_callback - INFO - Starting async job submit for node 69bdf731d54b5122738c6c3d 2026-03-21 01:42:13,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:42:13,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf731d54b5122738c6c3d HTTP/1.1" 200 None 2026-03-21 01:42:13,117 - lava_callback - DEBUG - Processing node 69bdf731d54b5122738c6c3d: baseline-arm64 2026-03-21 01:42:13,247 - lava_callback - WARNING - Failed to upload log 2026-03-21 01:42:13,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:42:13,500 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:42:13,502 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdf731d54b5122738c6c3d/lava-logs.yaml 2026-03-21 01:42:13,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:42:13,646 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:42:13,647 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdf731d54b5122738c6c3d/lava_callback.json.gz 2026-03-21 01:42:13,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:42:13,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf731d54b5122738c6c3d HTTP/1.1" 200 None 2026-03-21 01:42:13,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:42:13,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-21 01:42:13,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:42:13,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdf731d54b5122738c6c3d HTTP/1.1" 200 None 2026-03-21 01:42:13,798 - lava_callback - INFO - Completed processing callback for node 69bdf731d54b5122738c6c3d INFO: 172.21.0.1:59280 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38336 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:42:36,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:42:36,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:51078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51450 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51466 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:43:20,016 - lava_callback - INFO - Received callback for node 69bb54b0a1e2bfb1b334729e INFO: 172.21.0.1:60486 - "POST /node/69bb54b0a1e2bfb1b334729e HTTP/1.0" 202 Accepted 2026-03-21 01:43:20,021 - lava_callback - INFO - Starting async job submit for node 69bb54b0a1e2bfb1b334729e 2026-03-21 01:43:20,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:43:20,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb54b0a1e2bfb1b334729e HTTP/1.1" 200 None 2026-03-21 01:43:20,044 - lava_callback - DEBUG - Processing node 69bb54b0a1e2bfb1b334729e: baseline-arm64-baylibre 2026-03-21 01:43:20,044 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb54b0a1e2bfb1b334729e 2026-03-21 01:43:20,045 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:43:20,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb54b0a1e2bfb1b334729e HTTP/1.1" 200 None 2026-03-21 01:43:20,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:43:20,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-21 01:43:20,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:43:20,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb54b0a1e2bfb1b334729e HTTP/1.1" 200 None 2026-03-21 01:43:20,096 - lava_callback - ERROR - Error processing callback for node 69bb54b0a1e2bfb1b334729e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35156 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:44:10,381 - lava_callback - INFO - Received callback for node 69bdf35bd54b5122738c6970 INFO: 172.21.0.1:34320 - "POST /node/69bdf35bd54b5122738c6970 HTTP/1.0" 202 Accepted 2026-03-21 01:44:10,429 - lava_callback - INFO - Starting async job submit for node 69bdf35bd54b5122738c6970 2026-03-21 01:44:10,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:44:10,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf35bd54b5122738c6970 HTTP/1.1" 200 None 2026-03-21 01:44:10,473 - lava_callback - DEBUG - Processing node 69bdf35bd54b5122738c6970: baseline-arm-pengutronix 2026-03-21 01:44:10,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:44:10,721 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:44:10,723 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69bdf35bd54b5122738c6970/log.txt.gz 2026-03-21 01:44:10,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:44:10,951 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:44:10,953 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69bdf35bd54b5122738c6970/lava-logs.yaml 2026-03-21 01:44:10,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:44:11,068 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:44:11,070 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69bdf35bd54b5122738c6970/lava_callback.json.gz DEBUG: Setup failed for 69bdf35bd54b5122738c6970. Transitting job node result: incomplete -> fail 2026-03-21 01:44:11,119 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:44:11,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf35bd54b5122738c6970 HTTP/1.1" 200 None 2026-03-21 01:44:11,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:44:11,157 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-21 01:44:11,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:44:11,187 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdf35bd54b5122738c6970 HTTP/1.1" 200 None 2026-03-21 01:44:11,254 - lava_callback - INFO - Completed processing callback for node 69bdf35bd54b5122738c6970 INFO: 172.21.0.1:34336 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55320 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:44:36,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:56094 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:44:36,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-21 01:44:46,065 - lava_callback - INFO - Received callback for node 69bdf7d2d54b5122738c6c6e INFO: 172.21.0.1:45862 - "POST /node/69bdf7d2d54b5122738c6c6e HTTP/1.0" 202 Accepted 2026-03-21 01:44:46,071 - lava_callback - INFO - Starting async job submit for node 69bdf7d2d54b5122738c6c6e 2026-03-21 01:44:46,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:44:46,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf7d2d54b5122738c6c6e HTTP/1.1" 200 None 2026-03-21 01:44:46,115 - lava_callback - DEBUG - Processing node 69bdf7d2d54b5122738c6c6e: baseline-arm64 2026-03-21 01:44:46,245 - lava_callback - WARNING - Failed to upload log 2026-03-21 01:44:46,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:45876 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:44:46,521 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:44:46,523 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdf7d2d54b5122738c6c6e/lava-logs.yaml 2026-03-21 01:44:46,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:44:46,697 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:44:46,699 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdf7d2d54b5122738c6c6e/lava_callback.json.gz 2026-03-21 01:44:46,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:44:46,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf7d2d54b5122738c6c6e HTTP/1.1" 200 None 2026-03-21 01:44:46,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:44:46,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-21 01:44:46,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:44:46,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdf7d2d54b5122738c6c6e HTTP/1.1" 200 None 2026-03-21 01:44:46,878 - lava_callback - INFO - Completed processing callback for node 69bdf7d2d54b5122738c6c6e INFO: 172.21.0.1:46818 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:45:00,093 - lava_callback - INFO - Received callback for node 69bb5506a1e2bfb1b33472ae INFO: 172.21.0.1:40256 - "POST /node/69bb5506a1e2bfb1b33472ae HTTP/1.0" 202 Accepted 2026-03-21 01:45:00,098 - lava_callback - INFO - Starting async job submit for node 69bb5506a1e2bfb1b33472ae 2026-03-21 01:45:00,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:45:00,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb5506a1e2bfb1b33472ae HTTP/1.1" 200 None 2026-03-21 01:45:00,119 - lava_callback - DEBUG - Processing node 69bb5506a1e2bfb1b33472ae: baseline-arm64-baylibre 2026-03-21 01:45:00,119 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb5506a1e2bfb1b33472ae 2026-03-21 01:45:00,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:45:00,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb5506a1e2bfb1b33472ae HTTP/1.1" 200 None 2026-03-21 01:45:00,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:45:00,157 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-21 01:45:00,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:45:00,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb5506a1e2bfb1b33472ae HTTP/1.1" 200 None 2026-03-21 01:45:00,189 - lava_callback - ERROR - Error processing callback for node 69bb5506a1e2bfb1b33472ae: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40260 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:45:06,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:40264 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:45:06,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:55860 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52526 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52536 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:46:20,384 - lava_callback - INFO - Received callback for node 69bb5559a1e2bfb1b33472b8 INFO: 172.21.0.1:33534 - "POST /node/69bb5559a1e2bfb1b33472b8 HTTP/1.0" 202 Accepted 2026-03-21 01:46:20,389 - lava_callback - INFO - Starting async job submit for node 69bb5559a1e2bfb1b33472b8 2026-03-21 01:46:20,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:46:20,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb5559a1e2bfb1b33472b8 HTTP/1.1" 200 None 2026-03-21 01:46:20,405 - lava_callback - DEBUG - Processing node 69bb5559a1e2bfb1b33472b8: baseline-arm64-baylibre 2026-03-21 01:46:20,406 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb5559a1e2bfb1b33472b8 2026-03-21 01:46:20,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:46:20,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb5559a1e2bfb1b33472b8 HTTP/1.1" 200 None 2026-03-21 01:46:20,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:46:20,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-21 01:46:20,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:46:20,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb5559a1e2bfb1b33472b8 HTTP/1.1" 200 None 2026-03-21 01:46:20,465 - lava_callback - ERROR - Error processing callback for node 69bb5559a1e2bfb1b33472b8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53120 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:46:59,498 - lava_callback - INFO - Received callback for node 69bdf878d54b5122738c6c94 INFO: 172.21.0.1:53128 - "POST /node/69bdf878d54b5122738c6c94 HTTP/1.0" 202 Accepted 2026-03-21 01:46:59,504 - lava_callback - INFO - Starting async job submit for node 69bdf878d54b5122738c6c94 2026-03-21 01:46:59,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:46:59,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf878d54b5122738c6c94 HTTP/1.1" 200 None 2026-03-21 01:46:59,524 - lava_callback - DEBUG - Processing node 69bdf878d54b5122738c6c94: baseline-nfs-arm64 2026-03-21 01:46:59,525 - lava_callback - WARNING - Failed to upload log 2026-03-21 01:46:59,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:46:59,693 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:46:59,695 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69bdf878d54b5122738c6c94/lava-logs.yaml 2026-03-21 01:46:59,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:46:59,825 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:46:59,826 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69bdf878d54b5122738c6c94/lava_callback.json.gz 2026-03-21 01:46:59,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:46:59,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf878d54b5122738c6c94 HTTP/1.1" 200 None 2026-03-21 01:46:59,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:46:59,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-21 01:46:59,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:46:59,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdf878d54b5122738c6c94 HTTP/1.1" 200 None 2026-03-21 01:46:59,883 - lava_callback - INFO - Completed processing callback for node 69bdf878d54b5122738c6c94 INFO: 172.21.0.1:51098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51102 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:47:06,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:47:06,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:55704 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49046 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:47:40,331 - lava_callback - INFO - Received callback for node 69bb55b4a1e2bfb1b33472c3 INFO: 172.21.0.1:38742 - "POST /node/69bb55b4a1e2bfb1b33472c3 HTTP/1.0" 202 Accepted 2026-03-21 01:47:40,337 - lava_callback - INFO - Starting async job submit for node 69bb55b4a1e2bfb1b33472c3 2026-03-21 01:47:40,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:47:40,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb55b4a1e2bfb1b33472c3 HTTP/1.1" 200 None 2026-03-21 01:47:40,354 - lava_callback - DEBUG - Processing node 69bb55b4a1e2bfb1b33472c3: baseline-arm64-baylibre 2026-03-21 01:47:40,354 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb55b4a1e2bfb1b33472c3 2026-03-21 01:47:40,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:47:40,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb55b4a1e2bfb1b33472c3 HTTP/1.1" 200 None 2026-03-21 01:47:40,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:47:40,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-21 01:47:40,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:47:40,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb55b4a1e2bfb1b33472c3 HTTP/1.1" 200 None 2026-03-21 01:47:40,411 - lava_callback - ERROR - Error processing callback for node 69bb55b4a1e2bfb1b33472c3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36688 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36704 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:48:26,126 - lava_callback - INFO - Received callback for node 69bdf796d54b5122738c6c63 INFO: 172.21.0.1:60902 - "POST /node/69bdf796d54b5122738c6c63 HTTP/1.0" 202 Accepted 2026-03-21 01:48:26,135 - lava_callback - INFO - Starting async job submit for node 69bdf796d54b5122738c6c63 2026-03-21 01:48:26,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:48:26,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf796d54b5122738c6c63 HTTP/1.1" 200 None 2026-03-21 01:48:26,213 - lava_callback - DEBUG - Processing node 69bdf796d54b5122738c6c63: baseline-arm64-foundriesio INFO: 172.21.0.1:60914 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:48:27,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:48:27,523 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:48:27,525 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69bdf796d54b5122738c6c63/log.txt.gz 2026-03-21 01:48:28,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:48:28,483 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:48:28,486 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69bdf796d54b5122738c6c63/lava-logs.yaml 2026-03-21 01:48:28,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:48:28,711 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:48:28,713 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69bdf796d54b5122738c6c63/lava_callback.json.gz DEBUG: Setup failed for 69bdf796d54b5122738c6c63. Transitting job node result: incomplete -> fail 2026-03-21 01:48:28,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:48:28,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf796d54b5122738c6c63 HTTP/1.1" 200 None 2026-03-21 01:48:28,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:48:28,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-21 01:48:28,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:48:28,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdf796d54b5122738c6c63 HTTP/1.1" 200 None 2026-03-21 01:48:29,009 - lava_callback - INFO - Completed processing callback for node 69bdf796d54b5122738c6c63 INFO: 172.21.0.1:60542 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:48:36,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:48:36,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:32824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34890 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:49:13,669 - lava_callback - INFO - Received callback for node 69bb560ca1e2bfb1b33472cc INFO: 172.21.0.1:48000 - "POST /node/69bb560ca1e2bfb1b33472cc HTTP/1.0" 202 Accepted 2026-03-21 01:49:13,675 - lava_callback - INFO - Starting async job submit for node 69bb560ca1e2bfb1b33472cc 2026-03-21 01:49:13,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:49:13,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb560ca1e2bfb1b33472cc HTTP/1.1" 200 None 2026-03-21 01:49:13,693 - lava_callback - DEBUG - Processing node 69bb560ca1e2bfb1b33472cc: baseline-arm64-clabbe 2026-03-21 01:49:13,694 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb560ca1e2bfb1b33472cc 2026-03-21 01:49:13,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:49:13,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb560ca1e2bfb1b33472cc HTTP/1.1" 200 None 2026-03-21 01:49:13,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:49:13,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-21 01:49:13,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:49:13,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb560ca1e2bfb1b33472cc HTTP/1.1" 200 None 2026-03-21 01:49:13,746 - lava_callback - ERROR - Error processing callback for node 69bb560ca1e2bfb1b33472cc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48012 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58184 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58190 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:50:24,470 - lava_callback - INFO - Received callback for node 69bdf4a3d54b5122738c69ef INFO: 172.21.0.1:40400 - "POST /node/69bdf4a3d54b5122738c69ef HTTP/1.0" 202 Accepted 2026-03-21 01:50:24,480 - lava_callback - INFO - Starting async job submit for node 69bdf4a3d54b5122738c69ef 2026-03-21 01:50:24,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:50:24,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf4a3d54b5122738c69ef HTTP/1.1" 200 None 2026-03-21 01:50:24,546 - lava_callback - DEBUG - Processing node 69bdf4a3d54b5122738c69ef: baseline-nfs-arm64 2026-03-21 01:50:25,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:50:25,912 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:50:25,914 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69bdf4a3d54b5122738c69ef/log.txt.gz INFO: 172.21.0.1:40402 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:50:26,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:50:26,919 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:50:26,922 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69bdf4a3d54b5122738c69ef/lava-logs.yaml 2026-03-21 01:50:26,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:50:27,155 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:50:27,156 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69bdf4a3d54b5122738c69ef/lava_callback.json.gz DEBUG: Setup failed for 69bdf4a3d54b5122738c69ef. Transitting job node result: incomplete -> fail 2026-03-21 01:50:27,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:50:27,258 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf4a3d54b5122738c69ef HTTP/1.1" 200 None 2026-03-21 01:50:27,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:50:27,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-21 01:50:27,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:50:27,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdf4a3d54b5122738c69ef HTTP/1.1" 200 None 2026-03-21 01:50:27,389 - lava_callback - INFO - Completed processing callback for node 69bdf4a3d54b5122738c69ef 2026-03-21 01:50:33,783 - lava_callback - INFO - Received callback for node 69bb565fa1e2bfb1b33472d6 INFO: 172.21.0.1:53824 - "POST /node/69bb565fa1e2bfb1b33472d6 HTTP/1.0" 202 Accepted 2026-03-21 01:50:33,788 - lava_callback - INFO - Starting async job submit for node 69bb565fa1e2bfb1b33472d6 2026-03-21 01:50:33,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:50:33,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb565fa1e2bfb1b33472d6 HTTP/1.1" 200 None 2026-03-21 01:50:33,805 - lava_callback - DEBUG - Processing node 69bb565fa1e2bfb1b33472d6: baseline-arm64-clabbe 2026-03-21 01:50:33,805 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb565fa1e2bfb1b33472d6 2026-03-21 01:50:33,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:50:33,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb565fa1e2bfb1b33472d6 HTTP/1.1" 200 None 2026-03-21 01:50:33,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:50:33,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-21 01:50:33,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:50:33,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb565fa1e2bfb1b33472d6 HTTP/1.1" 200 None 2026-03-21 01:50:33,863 - lava_callback - ERROR - Error processing callback for node 69bb565fa1e2bfb1b33472d6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53826 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:50:36,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:50:36,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:60416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42430 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56492 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:51:53,686 - lava_callback - INFO - Received callback for node 69bb56b1a1e2bfb1b33472e7 INFO: 172.21.0.1:57502 - "POST /node/69bb56b1a1e2bfb1b33472e7 HTTP/1.0" 202 Accepted 2026-03-21 01:51:53,692 - lava_callback - INFO - Starting async job submit for node 69bb56b1a1e2bfb1b33472e7 2026-03-21 01:51:53,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:51:53,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb56b1a1e2bfb1b33472e7 HTTP/1.1" 200 None 2026-03-21 01:51:53,708 - lava_callback - DEBUG - Processing node 69bb56b1a1e2bfb1b33472e7: baseline-arm64-clabbe 2026-03-21 01:51:53,708 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb56b1a1e2bfb1b33472e7 2026-03-21 01:51:53,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:51:53,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb56b1a1e2bfb1b33472e7 HTTP/1.1" 200 None 2026-03-21 01:51:53,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:51:53,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-21 01:51:53,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:51:53,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb56b1a1e2bfb1b33472e7 HTTP/1.1" 200 None 2026-03-21 01:51:53,784 - lava_callback - ERROR - Error processing callback for node 69bb56b1a1e2bfb1b33472e7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49724 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53566 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:53:03,796 - lava_callback - INFO - Received callback for node 69bdf827d54b5122738c6c83 INFO: 172.21.0.1:46152 - "POST /node/69bdf827d54b5122738c6c83 HTTP/1.0" 202 Accepted 2026-03-21 01:53:03,803 - lava_callback - INFO - Starting async job submit for node 69bdf827d54b5122738c6c83 2026-03-21 01:53:03,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:53:03,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf827d54b5122738c6c83 HTTP/1.1" 200 None 2026-03-21 01:53:03,854 - lava_callback - DEBUG - Processing node 69bdf827d54b5122738c6c83: baseline-arm64 2026-03-21 01:53:04,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:53:04,393 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:53:04,394 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdf827d54b5122738c6c83/log.txt.gz 2026-03-21 01:53:04,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:53:04,834 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:53:04,836 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdf827d54b5122738c6c83/lava-logs.yaml 2026-03-21 01:53:04,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:46156 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:53:05,013 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:53:05,014 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdf827d54b5122738c6c83/lava_callback.json.gz DEBUG: Setup failed for 69bdf827d54b5122738c6c83. Transitting job node result: incomplete -> fail 2026-03-21 01:53:05,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:53:05,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf827d54b5122738c6c83 HTTP/1.1" 200 None 2026-03-21 01:53:05,119 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:53:05,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-21 01:53:05,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:53:05,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdf827d54b5122738c6c83 HTTP/1.1" 200 None 2026-03-21 01:53:05,240 - lava_callback - INFO - Completed processing callback for node 69bdf827d54b5122738c6c83 INFO: 172.21.0.1:46158 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:53:06,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:53:06,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-21 01:53:13,789 - lava_callback - INFO - Received callback for node 69bb5703a1e2bfb1b33472ef INFO: 172.21.0.1:45996 - "POST /node/69bb5703a1e2bfb1b33472ef HTTP/1.0" 202 Accepted 2026-03-21 01:53:13,795 - lava_callback - INFO - Starting async job submit for node 69bb5703a1e2bfb1b33472ef 2026-03-21 01:53:13,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:53:13,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb5703a1e2bfb1b33472ef HTTP/1.1" 200 None 2026-03-21 01:53:13,814 - lava_callback - DEBUG - Processing node 69bb5703a1e2bfb1b33472ef: baseline-arm64-clabbe 2026-03-21 01:53:13,814 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb5703a1e2bfb1b33472ef 2026-03-21 01:53:13,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:53:13,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb5703a1e2bfb1b33472ef HTTP/1.1" 200 None 2026-03-21 01:53:13,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:53:13,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-21 01:53:13,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:53:13,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb5703a1e2bfb1b33472ef HTTP/1.1" 200 None 2026-03-21 01:53:13,875 - lava_callback - ERROR - Error processing callback for node 69bb5703a1e2bfb1b33472ef: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46010 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60468 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:53:34,535 - lava_callback - INFO - Received callback for node 69bdf9d5d54b5122738c6ce0 INFO: 172.21.0.1:43456 - "POST /node/69bdf9d5d54b5122738c6ce0 HTTP/1.0" 202 Accepted 2026-03-21 01:53:34,541 - lava_callback - INFO - Starting async job submit for node 69bdf9d5d54b5122738c6ce0 2026-03-21 01:53:34,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:53:34,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf9d5d54b5122738c6ce0 HTTP/1.1" 200 None 2026-03-21 01:53:34,608 - lava_callback - DEBUG - Processing node 69bdf9d5d54b5122738c6ce0: baseline-arm64 2026-03-21 01:53:34,740 - lava_callback - WARNING - Failed to upload log 2026-03-21 01:53:34,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:53:34,965 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:53:34,967 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdf9d5d54b5122738c6ce0/lava-logs.yaml 2026-03-21 01:53:34,971 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:53:35,090 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:53:35,092 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdf9d5d54b5122738c6ce0/lava_callback.json.gz 2026-03-21 01:53:35,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:53:35,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf9d5d54b5122738c6ce0 HTTP/1.1" 200 None 2026-03-21 01:53:35,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:53:35,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-21 01:53:35,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:53:35,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdf9d5d54b5122738c6ce0 HTTP/1.1" 200 None 2026-03-21 01:53:35,246 - lava_callback - INFO - Completed processing callback for node 69bdf9d5d54b5122738c6ce0 INFO: 172.21.0.1:43472 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:53:36,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:53:36,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-21 01:53:37,886 - lava_callback - INFO - Received callback for node 69bdf4a9d54b5122738c6a00 INFO: 172.21.0.1:43482 - "POST /node/69bdf4a9d54b5122738c6a00 HTTP/1.0" 202 Accepted 2026-03-21 01:53:37,892 - lava_callback - INFO - Starting async job submit for node 69bdf4a9d54b5122738c6a00 2026-03-21 01:53:37,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:53:37,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf4a9d54b5122738c6a00 HTTP/1.1" 200 None 2026-03-21 01:53:37,932 - lava_callback - DEBUG - Processing node 69bdf4a9d54b5122738c6a00: baseline-nfs-arm64 2026-03-21 01:53:38,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:53:38,616 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:53:38,618 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69bdf4a9d54b5122738c6a00/log.txt.gz 2026-03-21 01:53:38,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:53:39,122 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:53:39,124 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69bdf4a9d54b5122738c6a00/lava-logs.yaml 2026-03-21 01:53:39,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:53:39,286 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:53:39,288 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69bdf4a9d54b5122738c6a00/lava_callback.json.gz 2026-03-21 01:53:39,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:53:39,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf4a9d54b5122738c6a00 HTTP/1.1" 200 None 2026-03-21 01:53:39,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:53:39,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-21 01:53:39,351 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:53:39,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdf4a9d54b5122738c6a00 HTTP/1.1" 200 None 2026-03-21 01:53:39,450 - lava_callback - INFO - Completed processing callback for node 69bdf4a9d54b5122738c6a00 INFO: 172.21.0.1:46704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44104 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:54:06,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:54:06,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:56960 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58096 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:54:33,796 - lava_callback - INFO - Received callback for node 69bb5755a1e2bfb1b33472f5 INFO: 172.21.0.1:53860 - "POST /node/69bb5755a1e2bfb1b33472f5 HTTP/1.0" 202 Accepted 2026-03-21 01:54:33,801 - lava_callback - INFO - Starting async job submit for node 69bb5755a1e2bfb1b33472f5 2026-03-21 01:54:33,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:54:33,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb5755a1e2bfb1b33472f5 HTTP/1.1" 200 None 2026-03-21 01:54:33,827 - lava_callback - DEBUG - Processing node 69bb5755a1e2bfb1b33472f5: baseline-arm64-clabbe 2026-03-21 01:54:33,827 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb5755a1e2bfb1b33472f5 2026-03-21 01:54:33,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:54:33,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb5755a1e2bfb1b33472f5 HTTP/1.1" 200 None 2026-03-21 01:54:33,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:54:33,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-21 01:54:33,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:54:33,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb5755a1e2bfb1b33472f5 HTTP/1.1" 200 None 2026-03-21 01:54:33,883 - lava_callback - ERROR - Error processing callback for node 69bb5755a1e2bfb1b33472f5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42816 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43206 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:55:53,865 - lava_callback - INFO - Received callback for node 69bb57a6a1e2bfb1b3347300 INFO: 172.21.0.1:43610 - "POST /node/69bb57a6a1e2bfb1b3347300 HTTP/1.0" 202 Accepted 2026-03-21 01:55:53,871 - lava_callback - INFO - Starting async job submit for node 69bb57a6a1e2bfb1b3347300 2026-03-21 01:55:53,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:55:53,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb57a6a1e2bfb1b3347300 HTTP/1.1" 200 None 2026-03-21 01:55:53,886 - lava_callback - DEBUG - Processing node 69bb57a6a1e2bfb1b3347300: baseline-arm64-clabbe 2026-03-21 01:55:53,886 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb57a6a1e2bfb1b3347300 2026-03-21 01:55:53,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:55:53,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb57a6a1e2bfb1b3347300 HTTP/1.1" 200 None 2026-03-21 01:55:53,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:55:53,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-21 01:55:53,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:55:53,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb57a6a1e2bfb1b3347300 HTTP/1.1" 200 None 2026-03-21 01:55:53,946 - lava_callback - ERROR - Error processing callback for node 69bb57a6a1e2bfb1b3347300: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48316 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51064 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:56:34,629 - lava_callback - INFO - Received callback for node 69bdfa9bd54b5122738c6d0d INFO: 172.21.0.1:52412 - "POST /node/69bdfa9bd54b5122738c6d0d HTTP/1.0" 202 Accepted 2026-03-21 01:56:34,635 - lava_callback - INFO - Starting async job submit for node 69bdfa9bd54b5122738c6d0d 2026-03-21 01:56:34,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:56:34,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdfa9bd54b5122738c6d0d HTTP/1.1" 200 None 2026-03-21 01:56:34,680 - lava_callback - DEBUG - Processing node 69bdfa9bd54b5122738c6d0d: baseline-arm64 2026-03-21 01:56:34,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:56:34,966 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:56:34,967 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdfa9bd54b5122738c6d0d/log.txt.gz 2026-03-21 01:56:35,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:56:35,236 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:56:35,237 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdfa9bd54b5122738c6d0d/lava-logs.yaml 2026-03-21 01:56:35,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 01:56:35,410 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 01:56:35,412 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdfa9bd54b5122738c6d0d/lava_callback.json.gz 2026-03-21 01:56:35,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:56:35,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdfa9bd54b5122738c6d0d HTTP/1.1" 200 None 2026-03-21 01:56:35,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:56:35,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-21 01:56:35,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:56:35,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdfa9bd54b5122738c6d0d HTTP/1.1" 200 None 2026-03-21 01:56:35,565 - lava_callback - INFO - Completed processing callback for node 69bdfa9bd54b5122738c6d0d INFO: 172.21.0.1:52422 - "GET / HTTP/1.0" 200 OK 2026-03-21 01:56:36,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 01:56:36,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:46366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34082 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57502 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40408 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53712 - "GET / HTTP/1.0" 200 OK 2026-03-21 02:00:01,066 - lava_callback - INFO - Received callback for node 69bb588fa1e2bfb1b3347319 INFO: 172.21.0.1:39524 - "POST /node/69bb588fa1e2bfb1b3347319 HTTP/1.0" 202 Accepted 2026-03-21 02:00:01,072 - lava_callback - INFO - Starting async job submit for node 69bb588fa1e2bfb1b3347319 2026-03-21 02:00:01,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:00:01,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb588fa1e2bfb1b3347319 HTTP/1.1" 200 None 2026-03-21 02:00:01,087 - lava_callback - DEBUG - Processing node 69bb588fa1e2bfb1b3347319: baseline-arm64-baylibre 2026-03-21 02:00:01,087 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb588fa1e2bfb1b3347319 2026-03-21 02:00:01,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:00:01,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb588fa1e2bfb1b3347319 HTTP/1.1" 200 None 2026-03-21 02:00:01,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:00:01,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-21 02:00:01,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:00:01,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb588fa1e2bfb1b3347319 HTTP/1.1" 200 None 2026-03-21 02:00:01,173 - lava_callback - ERROR - Error processing callback for node 69bb588fa1e2bfb1b3347319: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57270 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60930 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60934 - "GET / HTTP/1.0" 200 OK 2026-03-21 02:01:21,683 - lava_callback - INFO - Received callback for node 69bb58e1a1e2bfb1b3347322 INFO: 172.21.0.1:39180 - "POST /node/69bb58e1a1e2bfb1b3347322 HTTP/1.0" 202 Accepted 2026-03-21 02:01:21,688 - lava_callback - INFO - Starting async job submit for node 69bb58e1a1e2bfb1b3347322 2026-03-21 02:01:21,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:01:21,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb58e1a1e2bfb1b3347322 HTTP/1.1" 200 None 2026-03-21 02:01:21,705 - lava_callback - DEBUG - Processing node 69bb58e1a1e2bfb1b3347322: baseline-arm64-baylibre 2026-03-21 02:01:21,705 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb58e1a1e2bfb1b3347322 2026-03-21 02:01:21,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:01:21,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb58e1a1e2bfb1b3347322 HTTP/1.1" 200 None 2026-03-21 02:01:21,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:01:21,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-21 02:01:21,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:01:21,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb58e1a1e2bfb1b3347322 HTTP/1.1" 200 None 2026-03-21 02:01:21,758 - lava_callback - ERROR - Error processing callback for node 69bb58e1a1e2bfb1b3347322: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36940 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58978 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55950 - "GET / HTTP/1.0" 200 OK 2026-03-21 02:04:01,243 - lava_callback - INFO - Received callback for node 69bb5982a1e2bfb1b3347332 INFO: 172.21.0.1:38924 - "POST /node/69bb5982a1e2bfb1b3347332 HTTP/1.0" 202 Accepted 2026-03-21 02:04:01,249 - lava_callback - INFO - Starting async job submit for node 69bb5982a1e2bfb1b3347332 2026-03-21 02:04:01,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:04:01,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb5982a1e2bfb1b3347332 HTTP/1.1" 200 None 2026-03-21 02:04:01,265 - lava_callback - DEBUG - Processing node 69bb5982a1e2bfb1b3347332: baseline-arm64-baylibre 2026-03-21 02:04:01,266 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb5982a1e2bfb1b3347332 2026-03-21 02:04:01,266 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:04:01,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb5982a1e2bfb1b3347332 HTTP/1.1" 200 None 2026-03-21 02:04:01,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:04:01,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-21 02:04:01,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:04:01,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb5982a1e2bfb1b3347332 HTTP/1.1" 200 None 2026-03-21 02:04:01,333 - lava_callback - ERROR - Error processing callback for node 69bb5982a1e2bfb1b3347332: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43174 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51954 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46838 - "GET / HTTP/1.0" 200 OK 2026-03-21 02:06:05,416 - lava_callback - INFO - Received callback for node 69bdfbedd54b5122738c6d45 INFO: 172.21.0.1:46852 - "POST /node/69bdfbedd54b5122738c6d45 HTTP/1.0" 202 Accepted 2026-03-21 02:06:05,425 - lava_callback - INFO - Starting async job submit for node 69bdfbedd54b5122738c6d45 2026-03-21 02:06:05,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:06:05,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdfbedd54b5122738c6d45 HTTP/1.1" 200 None 2026-03-21 02:06:05,558 - lava_callback - DEBUG - Processing node 69bdfbedd54b5122738c6d45: baseline-arm64-foundriesio INFO: 172.21.0.1:46864 - "GET / HTTP/1.0" 200 OK 2026-03-21 02:06:06,678 - lava_callback - INFO - Received callback for node 69bdf915d54b5122738c6cb2 INFO: 172.21.0.1:46874 - "POST /node/69bdf915d54b5122738c6cb2 HTTP/1.0" 202 Accepted 2026-03-21 02:06:06,719 - lava_callback - INFO - Starting async job submit for node 69bdf915d54b5122738c6cb2 2026-03-21 02:06:06,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:06:06,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 02:06:06,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf915d54b5122738c6cb2 HTTP/1.1" 200 None 2026-03-21 02:06:06,771 - lava_callback - DEBUG - Processing node 69bdf915d54b5122738c6cb2: baseline-arm64 2026-03-21 02:06:07,015 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 02:06:07,063 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69bdfbedd54b5122738c6d45/log.txt.gz 2026-03-21 02:06:07,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 02:06:07,769 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 02:06:07,811 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdf915d54b5122738c6cb2/log.txt.gz 2026-03-21 02:06:08,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 02:06:08,239 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 02:06:08,490 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 02:06:08,492 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdf915d54b5122738c6cb2/lava-logs.yaml 2026-03-21 02:06:08,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 02:06:08,587 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 02:06:08,590 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69bdfbedd54b5122738c6d45/lava-logs.yaml 2026-03-21 02:06:08,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 02:06:08,718 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 02:06:08,720 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdf915d54b5122738c6cb2/lava_callback.json.gz 2026-03-21 02:06:08,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:06:08,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf915d54b5122738c6cb2 HTTP/1.1" 200 None 2026-03-21 02:06:08,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:06:08,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-21 02:06:08,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:06:08,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdf915d54b5122738c6cb2 HTTP/1.1" 200 None 2026-03-21 02:06:08,882 - lava_callback - INFO - Completed processing callback for node 69bdf915d54b5122738c6cb2 2026-03-21 02:06:08,883 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 02:06:08,885 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69bdfbedd54b5122738c6d45/lava_callback.json.gz DEBUG: 69bdfbedd54b5122738c6d45 Transitting job node result: pass -> incomplete 2026-03-21 02:06:08,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:06:08,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdfbedd54b5122738c6d45 HTTP/1.1" 200 None 2026-03-21 02:06:08,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:06:09,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-21 02:06:09,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:06:09,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdfbedd54b5122738c6d45 HTTP/1.1" 200 None 2026-03-21 02:06:09,072 - lava_callback - INFO - Completed processing callback for node 69bdfbedd54b5122738c6d45 INFO: 172.21.0.1:57016 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36556 - "GET / HTTP/1.0" 200 OK 2026-03-21 02:06:36,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:06:36,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:59274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36632 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44052 - "GET / HTTP/1.0" 200 OK 2026-03-21 02:07:51,683 - lava_callback - INFO - Received callback for node 69bdf961d54b5122738c6cbe INFO: 172.21.0.1:46236 - "POST /node/69bdf961d54b5122738c6cbe HTTP/1.0" 202 Accepted 2026-03-21 02:07:51,689 - lava_callback - INFO - Starting async job submit for node 69bdf961d54b5122738c6cbe 2026-03-21 02:07:51,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:07:51,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf961d54b5122738c6cbe HTTP/1.1" 200 None 2026-03-21 02:07:51,766 - lava_callback - DEBUG - Processing node 69bdf961d54b5122738c6cbe: ltp-smoke 2026-03-21 02:07:52,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 02:07:52,447 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 02:07:52,448 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69bdf961d54b5122738c6cbe/log.txt.gz 2026-03-21 02:07:52,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 02:07:52,979 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 02:07:52,981 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69bdf961d54b5122738c6cbe/lava-logs.yaml 2026-03-21 02:07:52,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 02:07:53,192 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 02:07:53,194 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69bdf961d54b5122738c6cbe/lava_callback.json.gz 2026-03-21 02:07:53,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:07:53,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf961d54b5122738c6cbe HTTP/1.1" 200 None 2026-03-21 02:07:53,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:07:53,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde35fd54b5122738c3d99 HTTP/1.1" 200 None 2026-03-21 02:07:53,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:07:53,374 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdf961d54b5122738c6cbe HTTP/1.1" 200 None 2026-03-21 02:07:53,486 - lava_callback - INFO - Completed processing callback for node 69bdf961d54b5122738c6cbe INFO: 172.21.0.1:46238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53520 - "GET / HTTP/1.0" 200 OK 2026-03-21 02:08:06,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:08:06,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-21 02:08:14,385 - lava_callback - INFO - Received callback for node 69bb5a7ba1e2bfb1b3347356 INFO: 172.21.0.1:52714 - "POST /node/69bb5a7ba1e2bfb1b3347356 HTTP/1.0" 202 Accepted 2026-03-21 02:08:14,391 - lava_callback - INFO - Starting async job submit for node 69bb5a7ba1e2bfb1b3347356 2026-03-21 02:08:14,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:08:14,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb5a7ba1e2bfb1b3347356 HTTP/1.1" 200 None 2026-03-21 02:08:14,410 - lava_callback - DEBUG - Processing node 69bb5a7ba1e2bfb1b3347356: baseline-arm64-clabbe 2026-03-21 02:08:14,410 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb5a7ba1e2bfb1b3347356 2026-03-21 02:08:14,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:08:14,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb5a7ba1e2bfb1b3347356 HTTP/1.1" 200 None 2026-03-21 02:08:14,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:08:14,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-21 02:08:14,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:08:14,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb5a7ba1e2bfb1b3347356 HTTP/1.1" 200 None 2026-03-21 02:08:14,466 - lava_callback - ERROR - Error processing callback for node 69bb5a7ba1e2bfb1b3347356: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52730 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50768 - "GET / HTTP/1.0" 200 OK 2026-03-21 02:08:54,878 - lava_callback - INFO - Received callback for node 69bdf9b2d54b5122738c6cd3 INFO: 172.21.0.1:37858 - "POST /node/69bdf9b2d54b5122738c6cd3 HTTP/1.0" 202 Accepted 2026-03-21 02:08:54,885 - lava_callback - INFO - Starting async job submit for node 69bdf9b2d54b5122738c6cd3 2026-03-21 02:08:54,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:08:54,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf9b2d54b5122738c6cd3 HTTP/1.1" 200 None 2026-03-21 02:08:54,959 - lava_callback - DEBUG - Processing node 69bdf9b2d54b5122738c6cd3: baseline-arm 2026-03-21 02:08:55,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 02:08:55,707 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 02:08:55,709 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69bdf9b2d54b5122738c6cd3/log.txt.gz 2026-03-21 02:08:56,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 02:08:56,322 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 02:08:56,325 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69bdf9b2d54b5122738c6cd3/lava-logs.yaml 2026-03-21 02:08:56,339 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 02:08:56,506 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 02:08:56,508 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69bdf9b2d54b5122738c6cd3/lava_callback.json.gz INFO: 172.21.0.1:37868 - "GET / HTTP/1.0" 200 OK DEBUG: Setup failed for 69bdf9b2d54b5122738c6cd3. Transitting job node result: incomplete -> fail 2026-03-21 02:08:56,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:08:56,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf9b2d54b5122738c6cd3 HTTP/1.1" 200 None 2026-03-21 02:08:56,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:08:56,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-21 02:08:56,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:08:56,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdf9b2d54b5122738c6cd3 HTTP/1.1" 200 None 2026-03-21 02:08:56,807 - lava_callback - INFO - Completed processing callback for node 69bdf9b2d54b5122738c6cd3 INFO: 172.21.0.1:38552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38556 - "GET / HTTP/1.0" 200 OK 2026-03-21 02:09:06,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:09:06,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:38012 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60596 - "GET / HTTP/1.0" 200 OK 2026-03-21 02:09:34,445 - lava_callback - INFO - Received callback for node 69bb5acea1e2bfb1b3347365 INFO: 172.21.0.1:33418 - "POST /node/69bb5acea1e2bfb1b3347365 HTTP/1.0" 202 Accepted 2026-03-21 02:09:34,451 - lava_callback - INFO - Starting async job submit for node 69bb5acea1e2bfb1b3347365 2026-03-21 02:09:34,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:09:34,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb5acea1e2bfb1b3347365 HTTP/1.1" 200 None 2026-03-21 02:09:34,466 - lava_callback - DEBUG - Processing node 69bb5acea1e2bfb1b3347365: baseline-arm64-clabbe 2026-03-21 02:09:34,466 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb5acea1e2bfb1b3347365 2026-03-21 02:09:34,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:09:34,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb5acea1e2bfb1b3347365 HTTP/1.1" 200 None 2026-03-21 02:09:34,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:09:34,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-21 02:09:34,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:09:34,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb5acea1e2bfb1b3347365 HTTP/1.1" 200 None 2026-03-21 02:09:34,529 - lava_callback - ERROR - Error processing callback for node 69bb5acea1e2bfb1b3347365: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59538 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53788 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41022 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34086 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41044 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59892 - "GET / HTTP/1.0" 200 OK 2026-03-21 02:14:51,507 - lava_callback - INFO - Received callback for node 69bdfa7fd54b5122738c6d03 INFO: 172.21.0.1:46456 - "POST /node/69bdfa7fd54b5122738c6d03 HTTP/1.0" 202 Accepted 2026-03-21 02:14:51,553 - lava_callback - INFO - Starting async job submit for node 69bdfa7fd54b5122738c6d03 2026-03-21 02:14:51,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:14:51,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdfa7fd54b5122738c6d03 HTTP/1.1" 200 None 2026-03-21 02:14:51,592 - lava_callback - DEBUG - Processing node 69bdfa7fd54b5122738c6d03: baseline-arm-pengutronix 2026-03-21 02:14:51,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 02:14:51,862 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 02:14:51,864 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69bdfa7fd54b5122738c6d03/log.txt.gz 2026-03-21 02:14:51,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 02:14:52,118 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 02:14:52,120 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69bdfa7fd54b5122738c6d03/lava-logs.yaml 2026-03-21 02:14:52,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 02:14:52,264 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 02:14:52,265 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69bdfa7fd54b5122738c6d03/lava_callback.json.gz DEBUG: Setup failed for 69bdfa7fd54b5122738c6d03. Transitting job node result: incomplete -> fail 2026-03-21 02:14:52,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:14:52,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdfa7fd54b5122738c6d03 HTTP/1.1" 200 None 2026-03-21 02:14:52,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:14:52,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-21 02:14:52,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:14:52,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdfa7fd54b5122738c6d03 HTTP/1.1" 200 None 2026-03-21 02:14:52,444 - lava_callback - INFO - Completed processing callback for node 69bdfa7fd54b5122738c6d03 2026-03-21 02:14:54,621 - lava_callback - INFO - Received callback for node 69bb5c15a1e2bfb1b33473c6 INFO: 172.21.0.1:46472 - "POST /node/69bb5c15a1e2bfb1b33473c6 HTTP/1.0" 202 Accepted 2026-03-21 02:14:54,626 - lava_callback - INFO - Starting async job submit for node 69bb5c15a1e2bfb1b33473c6 2026-03-21 02:14:54,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:14:54,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb5c15a1e2bfb1b33473c6 HTTP/1.1" 200 None 2026-03-21 02:14:54,647 - lava_callback - DEBUG - Processing node 69bb5c15a1e2bfb1b33473c6: baseline-arm64-clabbe 2026-03-21 02:14:54,648 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb5c15a1e2bfb1b33473c6 2026-03-21 02:14:54,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:14:54,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb5c15a1e2bfb1b33473c6 HTTP/1.1" 200 None 2026-03-21 02:14:54,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:14:54,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-21 02:14:54,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:14:54,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb5c15a1e2bfb1b33473c6 HTTP/1.1" 200 None 2026-03-21 02:14:54,706 - lava_callback - ERROR - Error processing callback for node 69bb5c15a1e2bfb1b33473c6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36428 - "GET / HTTP/1.0" 200 OK 2026-03-21 02:15:06,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:15:06,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:51208 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49966 - "GET / HTTP/1.0" 200 OK 2026-03-21 02:15:31,563 - lava_callback - INFO - Received callback for node 69bdfd92d54b5122738c6d8e INFO: 172.21.0.1:58944 - "POST /node/69bdfd92d54b5122738c6d8e HTTP/1.0" 202 Accepted 2026-03-21 02:15:31,570 - lava_callback - INFO - Starting async job submit for node 69bdfd92d54b5122738c6d8e 2026-03-21 02:15:31,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:15:31,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdfd92d54b5122738c6d8e HTTP/1.1" 200 None 2026-03-21 02:15:31,621 - lava_callback - DEBUG - Processing node 69bdfd92d54b5122738c6d8e: baseline-arm64 2026-03-21 02:15:31,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 02:15:32,122 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 02:15:32,124 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdfd92d54b5122738c6d8e/log.txt.gz 2026-03-21 02:15:32,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 02:15:32,524 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 02:15:32,526 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdfd92d54b5122738c6d8e/lava-logs.yaml 2026-03-21 02:15:32,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 02:15:32,702 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 02:15:32,704 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdfd92d54b5122738c6d8e/lava_callback.json.gz DEBUG: Setup failed for 69bdfd92d54b5122738c6d8e. Transitting job node result: incomplete -> fail 2026-03-21 02:15:32,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:15:32,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdfd92d54b5122738c6d8e HTTP/1.1" 200 None 2026-03-21 02:15:32,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:15:32,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-21 02:15:32,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:15:32,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdfd92d54b5122738c6d8e HTTP/1.1" 200 None 2026-03-21 02:15:32,906 - lava_callback - INFO - Completed processing callback for node 69bdfd92d54b5122738c6d8e INFO: 172.21.0.1:58956 - "GET / HTTP/1.0" 200 OK 2026-03-21 02:15:36,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:15:36,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:40894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51512 - "GET / HTTP/1.0" 200 OK 2026-03-21 02:16:14,622 - lava_callback - INFO - Received callback for node 69bb5c69a1e2bfb1b33473d8 INFO: 172.21.0.1:38554 - "POST /node/69bb5c69a1e2bfb1b33473d8 HTTP/1.0" 202 Accepted 2026-03-21 02:16:14,628 - lava_callback - INFO - Starting async job submit for node 69bb5c69a1e2bfb1b33473d8 2026-03-21 02:16:14,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:16:14,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb5c69a1e2bfb1b33473d8 HTTP/1.1" 200 None 2026-03-21 02:16:14,643 - lava_callback - DEBUG - Processing node 69bb5c69a1e2bfb1b33473d8: baseline-arm64-clabbe 2026-03-21 02:16:14,644 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb5c69a1e2bfb1b33473d8 2026-03-21 02:16:14,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:16:14,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb5c69a1e2bfb1b33473d8 HTTP/1.1" 200 None 2026-03-21 02:16:14,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:16:14,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-21 02:16:14,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:16:14,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb5c69a1e2bfb1b33473d8 HTTP/1.1" 200 None 2026-03-21 02:16:14,729 - lava_callback - ERROR - Error processing callback for node 69bb5c69a1e2bfb1b33473d8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38566 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59894 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54364 - "GET / HTTP/1.0" 200 OK 2026-03-21 02:17:34,741 - lava_callback - INFO - Received callback for node 69bb5cbaa1e2bfb1b33473dd INFO: 172.21.0.1:43132 - "POST /node/69bb5cbaa1e2bfb1b33473dd HTTP/1.0" 202 Accepted 2026-03-21 02:17:34,747 - lava_callback - INFO - Starting async job submit for node 69bb5cbaa1e2bfb1b33473dd 2026-03-21 02:17:34,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:17:34,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb5cbaa1e2bfb1b33473dd HTTP/1.1" 200 None 2026-03-21 02:17:34,762 - lava_callback - DEBUG - Processing node 69bb5cbaa1e2bfb1b33473dd: baseline-arm64-clabbe 2026-03-21 02:17:34,762 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb5cbaa1e2bfb1b33473dd 2026-03-21 02:17:34,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:17:34,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb5cbaa1e2bfb1b33473dd HTTP/1.1" 200 None 2026-03-21 02:17:34,779 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:17:34,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-21 02:17:34,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:17:34,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb5cbaa1e2bfb1b33473dd HTTP/1.1" 200 None 2026-03-21 02:17:34,818 - lava_callback - ERROR - Error processing callback for node 69bb5cbaa1e2bfb1b33473dd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59198 - "GET / HTTP/1.0" 200 OK 2026-03-21 02:18:01,906 - lava_callback - INFO - Received callback for node 69bdff8dd54b5122738c6de1 INFO: 172.21.0.1:48388 - "POST /node/69bdff8dd54b5122738c6de1 HTTP/1.0" 202 Accepted 2026-03-21 02:18:01,914 - lava_callback - INFO - Starting async job submit for node 69bdff8dd54b5122738c6de1 2026-03-21 02:18:01,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:18:01,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdff8dd54b5122738c6de1 HTTP/1.1" 200 None 2026-03-21 02:18:01,961 - lava_callback - DEBUG - Processing node 69bdff8dd54b5122738c6de1: baseline-arm64-foundriesio 2026-03-21 02:18:02,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 02:18:02,555 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 02:18:02,556 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69bdff8dd54b5122738c6de1/log.txt.gz 2026-03-21 02:18:02,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 02:18:03,048 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 02:18:03,050 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69bdff8dd54b5122738c6de1/lava-logs.yaml 2026-03-21 02:18:03,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 02:18:03,232 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 02:18:03,234 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69bdff8dd54b5122738c6de1/lava_callback.json.gz 2026-03-21 02:18:03,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:18:03,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdff8dd54b5122738c6de1 HTTP/1.1" 200 None 2026-03-21 02:18:03,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:18:03,292 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-21 02:18:03,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:18:03,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdff8dd54b5122738c6de1 HTTP/1.1" 200 None 2026-03-21 02:18:03,325 - lava_callback - INFO - Completed processing callback for node 69bdff8dd54b5122738c6de1 INFO: 172.21.0.1:48398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48402 - "GET / HTTP/1.0" 200 OK 2026-03-21 02:18:06,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:18:06,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:53526 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60658 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41302 - "GET / HTTP/1.0" 200 OK 2026-03-21 02:19:35,117 - lava_callback - INFO - Received callback for node 69bb5d31a1e2bfb1b33473ee INFO: 172.21.0.1:39402 - "POST /node/69bb5d31a1e2bfb1b33473ee HTTP/1.0" 202 Accepted 2026-03-21 02:19:35,123 - lava_callback - INFO - Starting async job submit for node 69bb5d31a1e2bfb1b33473ee 2026-03-21 02:19:35,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:19:35,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb5d31a1e2bfb1b33473ee HTTP/1.1" 200 None 2026-03-21 02:19:35,140 - lava_callback - DEBUG - Processing node 69bb5d31a1e2bfb1b33473ee: baseline-arm64-clabbe 2026-03-21 02:19:35,140 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb5d31a1e2bfb1b33473ee 2026-03-21 02:19:35,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:19:35,154 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb5d31a1e2bfb1b33473ee HTTP/1.1" 200 None 2026-03-21 02:19:35,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:19:35,195 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b3594d974c91e64560e4b2 HTTP/1.1" 200 None 2026-03-21 02:19:35,196 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:19:35,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb5d31a1e2bfb1b33473ee HTTP/1.1" 200 None 2026-03-21 02:19:35,222 - lava_callback - ERROR - Error processing callback for node 69bb5d31a1e2bfb1b33473ee: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45368 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57886 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46196 - "GET / HTTP/1.0" 200 OK 2026-03-21 02:21:59,126 - lava_callback - INFO - Received callback for node 69bdfe40d54b5122738c6daa INFO: 172.21.0.1:46212 - "POST /node/69bdfe40d54b5122738c6daa HTTP/1.0" 202 Accepted 2026-03-21 02:21:59,133 - lava_callback - INFO - Starting async job submit for node 69bdfe40d54b5122738c6daa 2026-03-21 02:21:59,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:21:59,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdfe40d54b5122738c6daa HTTP/1.1" 200 None 2026-03-21 02:21:59,174 - lava_callback - DEBUG - Processing node 69bdfe40d54b5122738c6daa: baseline-nfs-arm64 2026-03-21 02:21:59,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 02:21:59,844 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 02:21:59,846 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69bdfe40d54b5122738c6daa/log.txt.gz 2026-03-21 02:22:00,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 02:22:00,334 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 02:22:00,336 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69bdfe40d54b5122738c6daa/lava-logs.yaml 2026-03-21 02:22:00,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 02:22:00,548 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 02:22:00,549 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69bdfe40d54b5122738c6daa/lava_callback.json.gz 2026-03-21 02:22:00,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:22:00,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdfe40d54b5122738c6daa HTTP/1.1" 200 None 2026-03-21 02:22:00,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:22:00,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-21 02:22:00,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:22:00,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdfe40d54b5122738c6daa HTTP/1.1" 200 None 2026-03-21 02:22:00,685 - lava_callback - INFO - Completed processing callback for node 69bdfe40d54b5122738c6daa INFO: 172.21.0.1:39972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39976 - "GET / HTTP/1.0" 200 OK 2026-03-21 02:22:06,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:22:06,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:48270 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50500 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43672 - "GET / HTTP/1.0" 200 OK 2026-03-21 02:24:08,128 - lava_callback - INFO - Received callback for node 69bdfc9dd54b5122738c6d5d INFO: 172.21.0.1:43688 - "POST /node/69bdfc9dd54b5122738c6d5d HTTP/1.0" 202 Accepted 2026-03-21 02:24:08,136 - lava_callback - INFO - Starting async job submit for node 69bdfc9dd54b5122738c6d5d 2026-03-21 02:24:08,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:24:08,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdfc9dd54b5122738c6d5d HTTP/1.1" 200 None 2026-03-21 02:24:08,218 - lava_callback - DEBUG - Processing node 69bdfc9dd54b5122738c6d5d: baseline-nfs-arm64 2026-03-21 02:24:09,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 02:24:09,498 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 02:24:09,500 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69bdfc9dd54b5122738c6d5d/log.txt.gz 2026-03-21 02:24:10,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 02:24:10,372 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 02:24:10,374 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69bdfc9dd54b5122738c6d5d/lava-logs.yaml 2026-03-21 02:24:10,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 02:24:10,578 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 02:24:10,579 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69bdfc9dd54b5122738c6d5d/lava_callback.json.gz DEBUG: Setup failed for 69bdfc9dd54b5122738c6d5d. Transitting job node result: incomplete -> fail 2026-03-21 02:24:10,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:24:10,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdfc9dd54b5122738c6d5d HTTP/1.1" 200 None 2026-03-21 02:24:10,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:24:10,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-21 02:24:10,704 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:24:10,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdfc9dd54b5122738c6d5d HTTP/1.1" 200 None 2026-03-21 02:24:10,818 - lava_callback - INFO - Completed processing callback for node 69bdfc9dd54b5122738c6d5d INFO: 172.21.0.1:42124 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51976 - "GET / HTTP/1.0" 200 OK 2026-03-21 02:24:36,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:24:36,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:44566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36440 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49896 - "GET / HTTP/1.0" 200 OK 2026-03-21 02:25:34,384 - lava_callback - INFO - Received callback for node 69bdffc8d54b5122738c6dea INFO: 172.21.0.1:39948 - "POST /node/69bdffc8d54b5122738c6dea HTTP/1.0" 202 Accepted 2026-03-21 02:25:34,392 - lava_callback - INFO - Starting async job submit for node 69bdffc8d54b5122738c6dea 2026-03-21 02:25:34,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:25:34,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdffc8d54b5122738c6dea HTTP/1.1" 200 None 2026-03-21 02:25:34,442 - lava_callback - DEBUG - Processing node 69bdffc8d54b5122738c6dea: baseline-arm64 2026-03-21 02:25:34,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 02:25:34,973 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 02:25:34,975 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdffc8d54b5122738c6dea/log.txt.gz 2026-03-21 02:25:35,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 02:25:35,416 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 02:25:35,418 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdffc8d54b5122738c6dea/lava-logs.yaml 2026-03-21 02:25:35,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 02:25:35,627 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 02:25:35,628 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdffc8d54b5122738c6dea/lava_callback.json.gz DEBUG: Setup failed for 69bdffc8d54b5122738c6dea. Transitting job node result: incomplete -> fail 2026-03-21 02:25:35,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:25:35,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdffc8d54b5122738c6dea HTTP/1.1" 200 None 2026-03-21 02:25:35,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:25:35,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-21 02:25:35,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:25:35,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdffc8d54b5122738c6dea HTTP/1.1" 200 None 2026-03-21 02:25:35,850 - lava_callback - INFO - Completed processing callback for node 69bdffc8d54b5122738c6dea INFO: 172.21.0.1:39956 - "GET / HTTP/1.0" 200 OK 2026-03-21 02:25:36,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:25:36,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:48080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43754 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50560 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53570 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46776 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43388 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50742 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60034 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33700 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54948 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40434 - "GET / HTTP/1.0" 200 OK 2026-03-21 02:34:28,052 - lava_callback - INFO - Received callback for node 69bdff29d54b5122738c6dcd INFO: 172.21.0.1:40442 - "POST /node/69bdff29d54b5122738c6dcd HTTP/1.0" 202 Accepted 2026-03-21 02:34:28,059 - lava_callback - INFO - Starting async job submit for node 69bdff29d54b5122738c6dcd 2026-03-21 02:34:28,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:34:28,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdff29d54b5122738c6dcd HTTP/1.1" 200 None 2026-03-21 02:34:28,120 - lava_callback - DEBUG - Processing node 69bdff29d54b5122738c6dcd: baseline-arm64 2026-03-21 02:34:28,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 02:34:28,807 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 02:34:28,808 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdff29d54b5122738c6dcd/log.txt.gz 2026-03-21 02:34:29,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 02:34:29,294 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 02:34:29,296 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdff29d54b5122738c6dcd/lava-logs.yaml 2026-03-21 02:34:29,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 02:34:29,494 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 02:34:29,496 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69bdff29d54b5122738c6dcd/lava_callback.json.gz 2026-03-21 02:34:29,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:34:29,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdff29d54b5122738c6dcd HTTP/1.1" 200 None 2026-03-21 02:34:29,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:34:29,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-21 02:34:29,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:34:29,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdff29d54b5122738c6dcd HTTP/1.1" 200 None 2026-03-21 02:34:29,693 - lava_callback - INFO - Completed processing callback for node 69bdff29d54b5122738c6dcd INFO: 172.21.0.1:48728 - "GET / HTTP/1.0" 200 OK 2026-03-21 02:34:36,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:34:36,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:59582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55934 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59248 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58502 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58206 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49406 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43084 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36956 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39274 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57958 - "GET / HTTP/1.0" 200 OK 2026-03-21 02:43:03,382 - lava_callback - INFO - Received callback for node 69bb62a7a1e2bfb1b33474ae INFO: 172.21.0.1:52102 - "POST /node/69bb62a7a1e2bfb1b33474ae HTTP/1.0" 202 Accepted 2026-03-21 02:43:03,387 - lava_callback - INFO - Starting async job submit for node 69bb62a7a1e2bfb1b33474ae 2026-03-21 02:43:03,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:43:03,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb62a7a1e2bfb1b33474ae HTTP/1.1" 200 None 2026-03-21 02:43:03,427 - lava_callback - DEBUG - Processing node 69bb62a7a1e2bfb1b33474ae: baseline-arm-baylibre 2026-03-21 02:43:03,427 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb62a7a1e2bfb1b33474ae 2026-03-21 02:43:03,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:43:03,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb62a7a1e2bfb1b33474ae HTTP/1.1" 200 None 2026-03-21 02:43:03,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:43:03,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-21 02:43:03,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:43:03,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb62a7a1e2bfb1b33474ae HTTP/1.1" 200 None 2026-03-21 02:43:03,504 - lava_callback - ERROR - Error processing callback for node 69bb62a7a1e2bfb1b33474ae: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43036 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41318 - "GET / HTTP/1.0" 200 OK 2026-03-21 02:43:43,328 - lava_callback - INFO - Received callback for node 69bb62cba1e2bfb1b33474b8 INFO: 172.21.0.1:34920 - "POST /node/69bb62cba1e2bfb1b33474b8 HTTP/1.0" 202 Accepted 2026-03-21 02:43:43,334 - lava_callback - INFO - Starting async job submit for node 69bb62cba1e2bfb1b33474b8 2026-03-21 02:43:43,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:43:43,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb62cba1e2bfb1b33474b8 HTTP/1.1" 200 None 2026-03-21 02:43:43,350 - lava_callback - DEBUG - Processing node 69bb62cba1e2bfb1b33474b8: baseline-arm-baylibre 2026-03-21 02:43:43,350 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb62cba1e2bfb1b33474b8 2026-03-21 02:43:43,351 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:43:43,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb62cba1e2bfb1b33474b8 HTTP/1.1" 200 None 2026-03-21 02:43:43,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:43:43,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-21 02:43:43,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:43:43,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb62cba1e2bfb1b33474b8 HTTP/1.1" 200 None 2026-03-21 02:43:43,414 - lava_callback - ERROR - Error processing callback for node 69bb62cba1e2bfb1b33474b8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34144 - "GET / HTTP/1.0" 200 OK 2026-03-21 02:44:05,625 - lava_callback - INFO - Received callback for node 69be0153d54b5122738c6e0a INFO: 172.21.0.1:56014 - "POST /node/69be0153d54b5122738c6e0a HTTP/1.0" 202 Accepted 2026-03-21 02:44:05,633 - lava_callback - INFO - Starting async job submit for node 69be0153d54b5122738c6e0a INFO: 172.21.0.1:56026 - "GET / HTTP/1.0" 200 OK 2026-03-21 02:44:05,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:44:05,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be0153d54b5122738c6e0a HTTP/1.1" 200 None 2026-03-21 02:44:05,701 - lava_callback - DEBUG - Processing node 69be0153d54b5122738c6e0a: baseline-nfs-arm64 INFO: 172.21.0.1:56038 - "GET / HTTP/1.0" 200 OK 2026-03-21 02:44:06,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 02:44:06,987 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 02:44:06,989 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69be0153d54b5122738c6e0a/log.txt.gz 2026-03-21 02:44:07,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 02:44:07,879 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 02:44:07,882 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69be0153d54b5122738c6e0a/lava-logs.yaml 2026-03-21 02:44:07,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 02:44:08,096 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 02:44:08,098 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69be0153d54b5122738c6e0a/lava_callback.json.gz DEBUG: Setup failed for 69be0153d54b5122738c6e0a. Transitting job node result: incomplete -> fail 2026-03-21 02:44:08,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:44:08,210 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be0153d54b5122738c6e0a HTTP/1.1" 200 None 2026-03-21 02:44:08,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:44:08,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-21 02:44:08,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:44:08,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be0153d54b5122738c6e0a HTTP/1.1" 200 None 2026-03-21 02:44:08,348 - lava_callback - INFO - Completed processing callback for node 69be0153d54b5122738c6e0a INFO: 172.21.0.1:59286 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52944 - "GET / HTTP/1.0" 200 OK 2026-03-21 02:44:36,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:44:36,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:56712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43180 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43192 - "GET / HTTP/1.0" 200 OK 2026-03-21 02:45:23,502 - lava_callback - INFO - Received callback for node 69bb6340a1e2bfb1b33474d1 INFO: 172.21.0.1:59582 - "POST /node/69bb6340a1e2bfb1b33474d1 HTTP/1.0" 202 Accepted 2026-03-21 02:45:23,510 - lava_callback - INFO - Starting async job submit for node 69bb6340a1e2bfb1b33474d1 2026-03-21 02:45:23,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:45:23,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb6340a1e2bfb1b33474d1 HTTP/1.1" 200 None 2026-03-21 02:45:23,533 - lava_callback - DEBUG - Processing node 69bb6340a1e2bfb1b33474d1: baseline-arm-baylibre 2026-03-21 02:45:23,533 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb6340a1e2bfb1b33474d1 2026-03-21 02:45:23,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:45:23,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb6340a1e2bfb1b33474d1 HTTP/1.1" 200 None 2026-03-21 02:45:23,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:45:23,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-21 02:45:23,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:45:23,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb6340a1e2bfb1b33474d1 HTTP/1.1" 200 None 2026-03-21 02:45:23,604 - lava_callback - ERROR - Error processing callback for node 69bb6340a1e2bfb1b33474d1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58576 - "GET / HTTP/1.0" 200 OK 2026-03-21 02:46:03,468 - lava_callback - INFO - Received callback for node 69bb6365a1e2bfb1b33474dd INFO: 172.21.0.1:48664 - "POST /node/69bb6365a1e2bfb1b33474dd HTTP/1.0" 202 Accepted 2026-03-21 02:46:03,474 - lava_callback - INFO - Starting async job submit for node 69bb6365a1e2bfb1b33474dd 2026-03-21 02:46:03,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:46:03,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb6365a1e2bfb1b33474dd HTTP/1.1" 200 None 2026-03-21 02:46:03,490 - lava_callback - DEBUG - Processing node 69bb6365a1e2bfb1b33474dd: baseline-arm-baylibre 2026-03-21 02:46:03,490 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb6365a1e2bfb1b33474dd 2026-03-21 02:46:03,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:46:03,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb6365a1e2bfb1b33474dd HTTP/1.1" 200 None 2026-03-21 02:46:03,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:46:03,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-21 02:46:03,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:46:03,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb6365a1e2bfb1b33474dd HTTP/1.1" 200 None 2026-03-21 02:46:03,569 - lava_callback - ERROR - Error processing callback for node 69bb6365a1e2bfb1b33474dd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52184 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57046 - "GET / HTTP/1.0" 200 OK 2026-03-21 02:46:43,578 - lava_callback - INFO - Received callback for node 69bb638aa1e2bfb1b33474e7 INFO: 172.21.0.1:56042 - "POST /node/69bb638aa1e2bfb1b33474e7 HTTP/1.0" 202 Accepted 2026-03-21 02:46:43,584 - lava_callback - INFO - Starting async job submit for node 69bb638aa1e2bfb1b33474e7 2026-03-21 02:46:43,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:46:43,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb638aa1e2bfb1b33474e7 HTTP/1.1" 200 None 2026-03-21 02:46:43,600 - lava_callback - DEBUG - Processing node 69bb638aa1e2bfb1b33474e7: baseline-arm-baylibre 2026-03-21 02:46:43,600 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb638aa1e2bfb1b33474e7 2026-03-21 02:46:43,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:46:43,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb638aa1e2bfb1b33474e7 HTTP/1.1" 200 None 2026-03-21 02:46:43,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:46:43,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-21 02:46:43,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:46:43,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb638aa1e2bfb1b33474e7 HTTP/1.1" 200 None 2026-03-21 02:46:43,660 - lava_callback - ERROR - Error processing callback for node 69bb638aa1e2bfb1b33474e7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43650 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43652 - "GET / HTTP/1.0" 200 OK 2026-03-21 02:47:23,516 - lava_callback - INFO - Received callback for node 69bb63aea1e2bfb1b33474f8 INFO: 172.21.0.1:57368 - "POST /node/69bb63aea1e2bfb1b33474f8 HTTP/1.0" 202 Accepted 2026-03-21 02:47:23,521 - lava_callback - INFO - Starting async job submit for node 69bb63aea1e2bfb1b33474f8 2026-03-21 02:47:23,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:47:23,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb63aea1e2bfb1b33474f8 HTTP/1.1" 200 None 2026-03-21 02:47:23,543 - lava_callback - DEBUG - Processing node 69bb63aea1e2bfb1b33474f8: baseline-arm-baylibre 2026-03-21 02:47:23,543 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb63aea1e2bfb1b33474f8 2026-03-21 02:47:23,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:47:23,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb63aea1e2bfb1b33474f8 HTTP/1.1" 200 None 2026-03-21 02:47:23,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:47:23,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-21 02:47:23,573 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:47:23,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb63aea1e2bfb1b33474f8 HTTP/1.1" 200 None 2026-03-21 02:47:23,615 - lava_callback - ERROR - Error processing callback for node 69bb63aea1e2bfb1b33474f8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59214 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37234 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-21 02:49:16,172 - lava_callback - INFO - Received callback for node 69bb641ba1e2bfb1b3347516 INFO: 172.21.0.1:37248 - "POST /node/69bb641ba1e2bfb1b3347516 HTTP/1.0" 202 Accepted 2026-03-21 02:49:16,177 - lava_callback - INFO - Starting async job submit for node 69bb641ba1e2bfb1b3347516 2026-03-21 02:49:16,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:49:16,193 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb641ba1e2bfb1b3347516 HTTP/1.1" 200 None 2026-03-21 02:49:16,194 - lava_callback - DEBUG - Processing node 69bb641ba1e2bfb1b3347516: baseline-arm-clabbe 2026-03-21 02:49:16,194 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb641ba1e2bfb1b3347516 2026-03-21 02:49:16,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:49:16,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb641ba1e2bfb1b3347516 HTTP/1.1" 200 None 2026-03-21 02:49:16,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:49:16,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-21 02:49:16,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:49:16,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb641ba1e2bfb1b3347516 HTTP/1.1" 200 None 2026-03-21 02:49:16,297 - lava_callback - ERROR - Error processing callback for node 69bb641ba1e2bfb1b3347516: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58422 - "GET / HTTP/1.0" 200 OK 2026-03-21 02:49:56,200 - lava_callback - INFO - Received callback for node 69bb6440a1e2bfb1b3347520 INFO: 172.21.0.1:60708 - "POST /node/69bb6440a1e2bfb1b3347520 HTTP/1.0" 202 Accepted 2026-03-21 02:49:56,206 - lava_callback - INFO - Starting async job submit for node 69bb6440a1e2bfb1b3347520 2026-03-21 02:49:56,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:49:56,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb6440a1e2bfb1b3347520 HTTP/1.1" 200 None 2026-03-21 02:49:56,224 - lava_callback - DEBUG - Processing node 69bb6440a1e2bfb1b3347520: baseline-arm-clabbe 2026-03-21 02:49:56,224 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb6440a1e2bfb1b3347520 2026-03-21 02:49:56,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:49:56,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb6440a1e2bfb1b3347520 HTTP/1.1" 200 None 2026-03-21 02:49:56,239 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:49:56,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-21 02:49:56,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:49:56,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb6440a1e2bfb1b3347520 HTTP/1.1" 200 None 2026-03-21 02:49:56,280 - lava_callback - ERROR - Error processing callback for node 69bb6440a1e2bfb1b3347520: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50262 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-21 02:50:16,247 - lava_callback - INFO - Received callback for node 69bb6464a1e2bfb1b334752e INFO: 172.21.0.1:50266 - "POST /node/69bb6464a1e2bfb1b334752e HTTP/1.0" 202 Accepted 2026-03-21 02:50:16,253 - lava_callback - INFO - Starting async job submit for node 69bb6464a1e2bfb1b334752e 2026-03-21 02:50:16,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:50:16,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb6464a1e2bfb1b334752e HTTP/1.1" 200 None 2026-03-21 02:50:16,269 - lava_callback - DEBUG - Processing node 69bb6464a1e2bfb1b334752e: baseline-arm-clabbe 2026-03-21 02:50:16,269 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb6464a1e2bfb1b334752e 2026-03-21 02:50:16,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:50:16,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb6464a1e2bfb1b334752e HTTP/1.1" 200 None 2026-03-21 02:50:16,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:50:16,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-21 02:50:16,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:50:16,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb6464a1e2bfb1b334752e HTTP/1.1" 200 None 2026-03-21 02:50:16,355 - lava_callback - ERROR - Error processing callback for node 69bb6464a1e2bfb1b334752e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46770 - "GET / HTTP/1.0" 200 OK 2026-03-21 02:50:56,207 - lava_callback - INFO - Received callback for node 69bb6489a1e2bfb1b3347537 INFO: 172.21.0.1:50938 - "POST /node/69bb6489a1e2bfb1b3347537 HTTP/1.0" 202 Accepted 2026-03-21 02:50:56,212 - lava_callback - INFO - Starting async job submit for node 69bb6489a1e2bfb1b3347537 2026-03-21 02:50:56,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:50:56,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb6489a1e2bfb1b3347537 HTTP/1.1" 200 None 2026-03-21 02:50:56,229 - lava_callback - DEBUG - Processing node 69bb6489a1e2bfb1b3347537: baseline-arm-clabbe 2026-03-21 02:50:56,229 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb6489a1e2bfb1b3347537 2026-03-21 02:50:56,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:50:56,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb6489a1e2bfb1b3347537 HTTP/1.1" 200 None 2026-03-21 02:50:56,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:50:56,258 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-21 02:50:56,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:50:56,287 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb6489a1e2bfb1b3347537 HTTP/1.1" 200 None 2026-03-21 02:50:56,288 - lava_callback - ERROR - Error processing callback for node 69bb6489a1e2bfb1b3347537: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38150 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38634 - "GET / HTTP/1.0" 200 OK 2026-03-21 02:51:36,314 - lava_callback - INFO - Received callback for node 69bb64aea1e2bfb1b3347541 INFO: 172.21.0.1:47576 - "POST /node/69bb64aea1e2bfb1b3347541 HTTP/1.0" 202 Accepted 2026-03-21 02:51:36,320 - lava_callback - INFO - Starting async job submit for node 69bb64aea1e2bfb1b3347541 2026-03-21 02:51:36,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:51:36,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb64aea1e2bfb1b3347541 HTTP/1.1" 200 None 2026-03-21 02:51:36,359 - lava_callback - DEBUG - Processing node 69bb64aea1e2bfb1b3347541: baseline-arm-clabbe 2026-03-21 02:51:36,359 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb64aea1e2bfb1b3347541 2026-03-21 02:51:36,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:51:36,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb64aea1e2bfb1b3347541 HTTP/1.1" 200 None 2026-03-21 02:51:36,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:51:36,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-21 02:51:36,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:51:36,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb64aea1e2bfb1b3347541 HTTP/1.1" 200 None 2026-03-21 02:51:36,427 - lava_callback - ERROR - Error processing callback for node 69bb64aea1e2bfb1b3347541: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50534 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54596 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49520 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48378 - "GET / HTTP/1.0" 200 OK 2026-03-21 02:55:04,067 - lava_callback - INFO - Received callback for node 69bb6585a1e2bfb1b3347572 INFO: 172.21.0.1:34202 - "POST /node/69bb6585a1e2bfb1b3347572 HTTP/1.0" 202 Accepted 2026-03-21 02:55:04,073 - lava_callback - INFO - Starting async job submit for node 69bb6585a1e2bfb1b3347572 2026-03-21 02:55:04,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:55:04,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb6585a1e2bfb1b3347572 HTTP/1.1" 200 None 2026-03-21 02:55:04,095 - lava_callback - DEBUG - Processing node 69bb6585a1e2bfb1b3347572: baseline-arm64-baylibre 2026-03-21 02:55:04,096 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb6585a1e2bfb1b3347572 2026-03-21 02:55:04,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:55:04,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb6585a1e2bfb1b3347572 HTTP/1.1" 200 None 2026-03-21 02:55:04,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:55:04,129 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-21 02:55:04,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:55:04,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb6585a1e2bfb1b3347572 HTTP/1.1" 200 None 2026-03-21 02:55:04,158 - lava_callback - ERROR - Error processing callback for node 69bb6585a1e2bfb1b3347572: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39476 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57000 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43730 - "GET / HTTP/1.0" 200 OK 2026-03-21 02:57:04,091 - lava_callback - INFO - Received callback for node 69bb65fda1e2bfb1b334757f INFO: 172.21.0.1:51258 - "POST /node/69bb65fda1e2bfb1b334757f HTTP/1.0" 202 Accepted 2026-03-21 02:57:04,097 - lava_callback - INFO - Starting async job submit for node 69bb65fda1e2bfb1b334757f 2026-03-21 02:57:04,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:57:04,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb65fda1e2bfb1b334757f HTTP/1.1" 200 None 2026-03-21 02:57:04,112 - lava_callback - DEBUG - Processing node 69bb65fda1e2bfb1b334757f: baseline-arm64-baylibre 2026-03-21 02:57:04,112 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb65fda1e2bfb1b334757f 2026-03-21 02:57:04,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:57:04,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb65fda1e2bfb1b334757f HTTP/1.1" 200 None 2026-03-21 02:57:04,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:57:04,146 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-21 02:57:04,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:57:04,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb65fda1e2bfb1b334757f HTTP/1.1" 200 None 2026-03-21 02:57:04,173 - lava_callback - ERROR - Error processing callback for node 69bb65fda1e2bfb1b334757f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51810 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47128 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47138 - "GET / HTTP/1.0" 200 OK 2026-03-21 02:58:24,195 - lava_callback - INFO - Received callback for node 69bb664da1e2bfb1b334758a INFO: 172.21.0.1:37200 - "POST /node/69bb664da1e2bfb1b334758a HTTP/1.0" 202 Accepted 2026-03-21 02:58:24,200 - lava_callback - INFO - Starting async job submit for node 69bb664da1e2bfb1b334758a 2026-03-21 02:58:24,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:58:24,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb664da1e2bfb1b334758a HTTP/1.1" 200 None 2026-03-21 02:58:24,218 - lava_callback - DEBUG - Processing node 69bb664da1e2bfb1b334758a: baseline-arm64-baylibre 2026-03-21 02:58:24,218 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb664da1e2bfb1b334758a 2026-03-21 02:58:24,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:58:24,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb664da1e2bfb1b334758a HTTP/1.1" 200 None 2026-03-21 02:58:24,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:58:24,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-21 02:58:24,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:58:24,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb664da1e2bfb1b334758a HTTP/1.1" 200 None 2026-03-21 02:58:24,280 - lava_callback - ERROR - Error processing callback for node 69bb664da1e2bfb1b334758a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44710 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56810 - "GET / HTTP/1.0" 200 OK 2026-03-21 02:59:44,917 - lava_callback - INFO - Received callback for node 69bb669da1e2bfb1b3347596 INFO: 172.21.0.1:45802 - "POST /node/69bb669da1e2bfb1b3347596 HTTP/1.0" 202 Accepted 2026-03-21 02:59:44,922 - lava_callback - INFO - Starting async job submit for node 69bb669da1e2bfb1b3347596 2026-03-21 02:59:44,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:59:44,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb669da1e2bfb1b3347596 HTTP/1.1" 200 None 2026-03-21 02:59:44,938 - lava_callback - DEBUG - Processing node 69bb669da1e2bfb1b3347596: baseline-arm64-baylibre 2026-03-21 02:59:44,938 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb669da1e2bfb1b3347596 2026-03-21 02:59:44,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:59:44,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb669da1e2bfb1b3347596 HTTP/1.1" 200 None 2026-03-21 02:59:44,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:59:44,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-21 02:59:44,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 02:59:45,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb669da1e2bfb1b3347596 HTTP/1.1" 200 None 2026-03-21 02:59:45,032 - lava_callback - ERROR - Error processing callback for node 69bb669da1e2bfb1b3347596: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43386 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46330 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46340 - "GET / HTTP/1.0" 200 OK 2026-03-21 03:01:24,298 - lava_callback - INFO - Received callback for node 69bb66efa1e2bfb1b33475a3 INFO: 172.21.0.1:54956 - "POST /node/69bb66efa1e2bfb1b33475a3 HTTP/1.0" 202 Accepted 2026-03-21 03:01:24,303 - lava_callback - INFO - Starting async job submit for node 69bb66efa1e2bfb1b33475a3 2026-03-21 03:01:24,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:01:24,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb66efa1e2bfb1b33475a3 HTTP/1.1" 200 None 2026-03-21 03:01:24,342 - lava_callback - DEBUG - Processing node 69bb66efa1e2bfb1b33475a3: baseline-arm64-baylibre 2026-03-21 03:01:24,343 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb66efa1e2bfb1b33475a3 2026-03-21 03:01:24,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:01:24,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb66efa1e2bfb1b33475a3 HTTP/1.1" 200 None 2026-03-21 03:01:24,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:01:24,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-21 03:01:24,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:01:24,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb66efa1e2bfb1b33475a3 HTTP/1.1" 200 None 2026-03-21 03:01:24,406 - lava_callback - ERROR - Error processing callback for node 69bb66efa1e2bfb1b33475a3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46464 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53348 - "GET / HTTP/1.0" 200 OK 2026-03-21 03:02:44,378 - lava_callback - INFO - Received callback for node 69bb6741a1e2bfb1b33475ae INFO: 172.21.0.1:39378 - "POST /node/69bb6741a1e2bfb1b33475ae HTTP/1.0" 202 Accepted 2026-03-21 03:02:44,383 - lava_callback - INFO - Starting async job submit for node 69bb6741a1e2bfb1b33475ae 2026-03-21 03:02:44,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:02:44,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb6741a1e2bfb1b33475ae HTTP/1.1" 200 None 2026-03-21 03:02:44,399 - lava_callback - DEBUG - Processing node 69bb6741a1e2bfb1b33475ae: baseline-arm64-baylibre 2026-03-21 03:02:44,399 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb6741a1e2bfb1b33475ae 2026-03-21 03:02:44,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:02:44,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb6741a1e2bfb1b33475ae HTTP/1.1" 200 None 2026-03-21 03:02:44,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:02:44,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-21 03:02:44,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:02:44,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb6741a1e2bfb1b33475ae HTTP/1.1" 200 None 2026-03-21 03:02:44,458 - lava_callback - ERROR - Error processing callback for node 69bb6741a1e2bfb1b33475ae: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53382 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56206 - "GET / HTTP/1.0" 200 OK 2026-03-21 03:04:04,525 - lava_callback - INFO - Received callback for node 69bb6793a1e2bfb1b33475bd INFO: 172.21.0.1:46438 - "POST /node/69bb6793a1e2bfb1b33475bd HTTP/1.0" 202 Accepted 2026-03-21 03:04:04,530 - lava_callback - INFO - Starting async job submit for node 69bb6793a1e2bfb1b33475bd 2026-03-21 03:04:04,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:04:04,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb6793a1e2bfb1b33475bd HTTP/1.1" 200 None 2026-03-21 03:04:04,548 - lava_callback - DEBUG - Processing node 69bb6793a1e2bfb1b33475bd: baseline-arm64-baylibre 2026-03-21 03:04:04,548 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb6793a1e2bfb1b33475bd 2026-03-21 03:04:04,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:04:04,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb6793a1e2bfb1b33475bd HTTP/1.1" 200 None 2026-03-21 03:04:04,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:04:04,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-21 03:04:04,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:04:04,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb6793a1e2bfb1b33475bd HTTP/1.1" 200 None 2026-03-21 03:04:04,627 - lava_callback - ERROR - Error processing callback for node 69bb6793a1e2bfb1b33475bd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-21 03:04:05,841 - lava_callback - INFO - Received callback for node 69be0600d54b5122738c6e30 INFO: 172.21.0.1:46440 - "POST /node/69be0600d54b5122738c6e30 HTTP/1.0" 202 Accepted 2026-03-21 03:04:05,849 - lava_callback - INFO - Starting async job submit for node 69be0600d54b5122738c6e30 2026-03-21 03:04:05,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:04:05,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be0600d54b5122738c6e30 HTTP/1.1" 200 None 2026-03-21 03:04:05,907 - lava_callback - DEBUG - Processing node 69be0600d54b5122738c6e30: baseline-nfs-arm64 INFO: 172.21.0.1:46452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46456 - "GET / HTTP/1.0" 200 OK 2026-03-21 03:04:06,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 03:04:07,212 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 03:04:07,213 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69be0600d54b5122738c6e30/log.txt.gz 2026-03-21 03:04:08,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 03:04:08,348 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 03:04:08,350 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69be0600d54b5122738c6e30/lava-logs.yaml 2026-03-21 03:04:08,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 03:04:08,551 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 03:04:08,553 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69be0600d54b5122738c6e30/lava_callback.json.gz DEBUG: Setup failed for 69be0600d54b5122738c6e30. Transitting job node result: incomplete -> fail 2026-03-21 03:04:08,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:04:08,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be0600d54b5122738c6e30 HTTP/1.1" 200 None 2026-03-21 03:04:08,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:04:08,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-21 03:04:08,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:04:08,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be0600d54b5122738c6e30 HTTP/1.1" 200 None 2026-03-21 03:04:08,793 - lava_callback - INFO - Completed processing callback for node 69be0600d54b5122738c6e30 INFO: 172.21.0.1:50608 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33146 - "GET / HTTP/1.0" 200 OK 2026-03-21 03:04:36,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:04:37,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:53856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35218 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47722 - "GET / HTTP/1.0" 200 OK 2026-03-21 03:05:24,516 - lava_callback - INFO - Received callback for node 69bb67e4a1e2bfb1b33475ce INFO: 172.21.0.1:47732 - "POST /node/69bb67e4a1e2bfb1b33475ce HTTP/1.0" 202 Accepted 2026-03-21 03:05:24,522 - lava_callback - INFO - Starting async job submit for node 69bb67e4a1e2bfb1b33475ce 2026-03-21 03:05:24,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:05:24,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb67e4a1e2bfb1b33475ce HTTP/1.1" 200 None 2026-03-21 03:05:24,544 - lava_callback - DEBUG - Processing node 69bb67e4a1e2bfb1b33475ce: baseline-arm64-baylibre 2026-03-21 03:05:24,545 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb67e4a1e2bfb1b33475ce 2026-03-21 03:05:24,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:05:24,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb67e4a1e2bfb1b33475ce HTTP/1.1" 200 None 2026-03-21 03:05:24,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:05:24,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-21 03:05:24,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:05:24,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb67e4a1e2bfb1b33475ce HTTP/1.1" 200 None 2026-03-21 03:05:24,597 - lava_callback - ERROR - Error processing callback for node 69bb67e4a1e2bfb1b33475ce: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45080 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60200 - "GET / HTTP/1.0" 200 OK 2026-03-21 03:06:44,702 - lava_callback - INFO - Received callback for node 69bb6835a1e2bfb1b33475d5 INFO: 172.21.0.1:44594 - "POST /node/69bb6835a1e2bfb1b33475d5 HTTP/1.0" 202 Accepted 2026-03-21 03:06:44,708 - lava_callback - INFO - Starting async job submit for node 69bb6835a1e2bfb1b33475d5 2026-03-21 03:06:44,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:06:44,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb6835a1e2bfb1b33475d5 HTTP/1.1" 200 None 2026-03-21 03:06:44,724 - lava_callback - DEBUG - Processing node 69bb6835a1e2bfb1b33475d5: baseline-arm64-baylibre 2026-03-21 03:06:44,724 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb6835a1e2bfb1b33475d5 2026-03-21 03:06:44,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:06:44,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb6835a1e2bfb1b33475d5 HTTP/1.1" 200 None 2026-03-21 03:06:44,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:06:44,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-21 03:06:44,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:06:44,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb6835a1e2bfb1b33475d5 HTTP/1.1" 200 None 2026-03-21 03:06:44,792 - lava_callback - ERROR - Error processing callback for node 69bb6835a1e2bfb1b33475d5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52980 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46104 - "GET / HTTP/1.0" 200 OK 2026-03-21 03:08:04,680 - lava_callback - INFO - Received callback for node 69bb6883a1e2bfb1b33475e3 INFO: 172.21.0.1:44426 - "POST /node/69bb6883a1e2bfb1b33475e3 HTTP/1.0" 202 Accepted 2026-03-21 03:08:04,685 - lava_callback - INFO - Starting async job submit for node 69bb6883a1e2bfb1b33475e3 2026-03-21 03:08:04,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:08:04,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb6883a1e2bfb1b33475e3 HTTP/1.1" 200 None 2026-03-21 03:08:04,703 - lava_callback - DEBUG - Processing node 69bb6883a1e2bfb1b33475e3: baseline-arm64-baylibre 2026-03-21 03:08:04,703 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb6883a1e2bfb1b33475e3 2026-03-21 03:08:04,704 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:08:04,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb6883a1e2bfb1b33475e3 HTTP/1.1" 200 None 2026-03-21 03:08:04,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:08:04,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-21 03:08:04,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:08:04,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb6883a1e2bfb1b33475e3 HTTP/1.1" 200 None 2026-03-21 03:08:04,798 - lava_callback - ERROR - Error processing callback for node 69bb6883a1e2bfb1b33475e3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49722 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53424 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53430 - "GET / HTTP/1.0" 200 OK 2026-03-21 03:09:24,927 - lava_callback - INFO - Received callback for node 69bb68d7a1e2bfb1b33475f2 INFO: 172.21.0.1:57210 - "POST /node/69bb68d7a1e2bfb1b33475f2 HTTP/1.0" 202 Accepted 2026-03-21 03:09:24,933 - lava_callback - INFO - Starting async job submit for node 69bb68d7a1e2bfb1b33475f2 2026-03-21 03:09:24,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:09:24,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb68d7a1e2bfb1b33475f2 HTTP/1.1" 200 None 2026-03-21 03:09:24,949 - lava_callback - DEBUG - Processing node 69bb68d7a1e2bfb1b33475f2: baseline-arm64-baylibre 2026-03-21 03:09:24,949 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb68d7a1e2bfb1b33475f2 2026-03-21 03:09:24,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:09:24,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb68d7a1e2bfb1b33475f2 HTTP/1.1" 200 None 2026-03-21 03:09:24,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:09:24,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-21 03:09:24,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:09:25,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb68d7a1e2bfb1b33475f2 HTTP/1.1" 200 None 2026-03-21 03:09:25,030 - lava_callback - ERROR - Error processing callback for node 69bb68d7a1e2bfb1b33475f2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37718 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38182 - "GET / HTTP/1.0" 200 OK 2026-03-21 03:10:44,809 - lava_callback - INFO - Received callback for node 69bb692ea1e2bfb1b33475ff INFO: 172.21.0.1:42840 - "POST /node/69bb692ea1e2bfb1b33475ff HTTP/1.0" 202 Accepted 2026-03-21 03:10:44,814 - lava_callback - INFO - Starting async job submit for node 69bb692ea1e2bfb1b33475ff 2026-03-21 03:10:44,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:10:44,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb692ea1e2bfb1b33475ff HTTP/1.1" 200 None 2026-03-21 03:10:44,831 - lava_callback - DEBUG - Processing node 69bb692ea1e2bfb1b33475ff: baseline-arm64-baylibre 2026-03-21 03:10:44,831 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb692ea1e2bfb1b33475ff 2026-03-21 03:10:44,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:10:44,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb692ea1e2bfb1b33475ff HTTP/1.1" 200 None 2026-03-21 03:10:44,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:10:44,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-21 03:10:44,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:10:44,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb692ea1e2bfb1b33475ff HTTP/1.1" 200 None 2026-03-21 03:10:44,891 - lava_callback - ERROR - Error processing callback for node 69bb692ea1e2bfb1b33475ff: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39164 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56578 - "GET / HTTP/1.0" 200 OK 2026-03-21 03:12:05,024 - lava_callback - INFO - Received callback for node 69bb697fa1e2bfb1b334760b INFO: 172.21.0.1:38294 - "POST /node/69bb697fa1e2bfb1b334760b HTTP/1.0" 202 Accepted 2026-03-21 03:12:05,030 - lava_callback - INFO - Starting async job submit for node 69bb697fa1e2bfb1b334760b 2026-03-21 03:12:05,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:12:05,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb697fa1e2bfb1b334760b HTTP/1.1" 200 None 2026-03-21 03:12:05,046 - lava_callback - DEBUG - Processing node 69bb697fa1e2bfb1b334760b: baseline-arm64-baylibre 2026-03-21 03:12:05,046 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb697fa1e2bfb1b334760b 2026-03-21 03:12:05,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:12:05,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb697fa1e2bfb1b334760b HTTP/1.1" 200 None 2026-03-21 03:12:05,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:12:05,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-21 03:12:05,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:12:05,130 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb697fa1e2bfb1b334760b HTTP/1.1" 200 None 2026-03-21 03:12:05,132 - lava_callback - ERROR - Error processing callback for node 69bb697fa1e2bfb1b334760b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46718 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47578 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47580 - "GET / HTTP/1.0" 200 OK 2026-03-21 03:13:24,976 - lava_callback - INFO - Received callback for node 69bb69d1a1e2bfb1b3347618 INFO: 172.21.0.1:55148 - "POST /node/69bb69d1a1e2bfb1b3347618 HTTP/1.0" 202 Accepted 2026-03-21 03:13:24,981 - lava_callback - INFO - Starting async job submit for node 69bb69d1a1e2bfb1b3347618 2026-03-21 03:13:24,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:13:24,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb69d1a1e2bfb1b3347618 HTTP/1.1" 200 None 2026-03-21 03:13:24,998 - lava_callback - DEBUG - Processing node 69bb69d1a1e2bfb1b3347618: baseline-arm64-baylibre 2026-03-21 03:13:24,998 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb69d1a1e2bfb1b3347618 2026-03-21 03:13:24,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:13:25,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb69d1a1e2bfb1b3347618 HTTP/1.1" 200 None 2026-03-21 03:13:25,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:13:25,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-21 03:13:25,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:13:25,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb69d1a1e2bfb1b3347618 HTTP/1.1" 200 None 2026-03-21 03:13:25,059 - lava_callback - ERROR - Error processing callback for node 69bb69d1a1e2bfb1b3347618: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38202 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49250 - "GET / HTTP/1.0" 200 OK 2026-03-21 03:15:05,105 - lava_callback - INFO - Received callback for node 69bb6a25a1e2bfb1b334762b INFO: 172.21.0.1:53202 - "POST /node/69bb6a25a1e2bfb1b334762b HTTP/1.0" 202 Accepted 2026-03-21 03:15:05,110 - lava_callback - INFO - Starting async job submit for node 69bb6a25a1e2bfb1b334762b 2026-03-21 03:15:05,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:15:05,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb6a25a1e2bfb1b334762b HTTP/1.1" 200 None 2026-03-21 03:15:05,127 - lava_callback - DEBUG - Processing node 69bb6a25a1e2bfb1b334762b: baseline-arm64-baylibre 2026-03-21 03:15:05,127 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb6a25a1e2bfb1b334762b 2026-03-21 03:15:05,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:15:05,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb6a25a1e2bfb1b334762b HTTP/1.1" 200 None 2026-03-21 03:15:05,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:15:05,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-21 03:15:05,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:15:05,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb6a25a1e2bfb1b334762b HTTP/1.1" 200 None 2026-03-21 03:15:05,199 - lava_callback - ERROR - Error processing callback for node 69bb6a25a1e2bfb1b334762b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49686 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44732 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44746 - "GET / HTTP/1.0" 200 OK 2026-03-21 03:16:17,398 - lava_callback - INFO - Received callback for node 69bb6a7ca1e2bfb1b334763a INFO: 172.21.0.1:44750 - "POST /node/69bb6a7ca1e2bfb1b334763a HTTP/1.0" 202 Accepted 2026-03-21 03:16:17,404 - lava_callback - INFO - Starting async job submit for node 69bb6a7ca1e2bfb1b334763a 2026-03-21 03:16:17,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:16:17,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb6a7ca1e2bfb1b334763a HTTP/1.1" 200 None 2026-03-21 03:16:17,425 - lava_callback - DEBUG - Processing node 69bb6a7ca1e2bfb1b334763a: baseline-arm64-clabbe 2026-03-21 03:16:17,425 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb6a7ca1e2bfb1b334763a 2026-03-21 03:16:17,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:16:17,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb6a7ca1e2bfb1b334763a HTTP/1.1" 200 None 2026-03-21 03:16:17,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:16:17,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-21 03:16:17,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:16:17,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb6a7ca1e2bfb1b334763a HTTP/1.1" 200 None 2026-03-21 03:16:17,494 - lava_callback - ERROR - Error processing callback for node 69bb6a7ca1e2bfb1b334763a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46758 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34378 - "GET / HTTP/1.0" 200 OK 2026-03-21 03:17:45,249 - lava_callback - INFO - Received callback for node 69bb6ac7a1e2bfb1b3347640 INFO: 172.21.0.1:58144 - "POST /node/69bb6ac7a1e2bfb1b3347640 HTTP/1.0" 202 Accepted 2026-03-21 03:17:45,254 - lava_callback - INFO - Starting async job submit for node 69bb6ac7a1e2bfb1b3347640 2026-03-21 03:17:45,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:17:45,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb6ac7a1e2bfb1b3347640 HTTP/1.1" 200 None 2026-03-21 03:17:45,273 - lava_callback - DEBUG - Processing node 69bb6ac7a1e2bfb1b3347640: baseline-arm64-baylibre 2026-03-21 03:17:45,274 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb6ac7a1e2bfb1b3347640 2026-03-21 03:17:45,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:17:45,287 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb6ac7a1e2bfb1b3347640 HTTP/1.1" 200 None 2026-03-21 03:17:45,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:17:45,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-21 03:17:45,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:17:45,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb6ac7a1e2bfb1b3347640 HTTP/1.1" 200 None 2026-03-21 03:17:45,328 - lava_callback - ERROR - Error processing callback for node 69bb6ac7a1e2bfb1b3347640: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43812 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59134 - "GET / HTTP/1.0" 200 OK 2026-03-21 03:18:57,499 - lava_callback - INFO - Received callback for node 69bb6b1fa1e2bfb1b334764f INFO: 172.21.0.1:59138 - "POST /node/69bb6b1fa1e2bfb1b334764f HTTP/1.0" 202 Accepted 2026-03-21 03:18:57,504 - lava_callback - INFO - Starting async job submit for node 69bb6b1fa1e2bfb1b334764f 2026-03-21 03:18:57,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:18:57,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb6b1fa1e2bfb1b334764f HTTP/1.1" 200 None 2026-03-21 03:18:57,530 - lava_callback - DEBUG - Processing node 69bb6b1fa1e2bfb1b334764f: baseline-arm64-clabbe 2026-03-21 03:18:57,530 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb6b1fa1e2bfb1b334764f 2026-03-21 03:18:57,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:18:57,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb6b1fa1e2bfb1b334764f HTTP/1.1" 200 None 2026-03-21 03:18:57,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:18:57,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-21 03:18:57,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:18:57,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb6b1fa1e2bfb1b334764f HTTP/1.1" 200 None 2026-03-21 03:18:57,612 - lava_callback - ERROR - Error processing callback for node 69bb6b1fa1e2bfb1b334764f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53544 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36432 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46782 - "GET / HTTP/1.0" 200 OK 2026-03-21 03:20:37,580 - lava_callback - INFO - Received callback for node 69bb6b72a1e2bfb1b3347659 INFO: 172.21.0.1:46796 - "POST /node/69bb6b72a1e2bfb1b3347659 HTTP/1.0" 202 Accepted 2026-03-21 03:20:37,586 - lava_callback - INFO - Starting async job submit for node 69bb6b72a1e2bfb1b3347659 2026-03-21 03:20:37,586 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:20:37,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb6b72a1e2bfb1b3347659 HTTP/1.1" 200 None 2026-03-21 03:20:37,602 - lava_callback - DEBUG - Processing node 69bb6b72a1e2bfb1b3347659: baseline-arm64-clabbe 2026-03-21 03:20:37,602 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb6b72a1e2bfb1b3347659 2026-03-21 03:20:37,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:20:37,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb6b72a1e2bfb1b3347659 HTTP/1.1" 200 None 2026-03-21 03:20:37,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:20:37,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-21 03:20:37,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:20:37,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb6b72a1e2bfb1b3347659 HTTP/1.1" 200 None 2026-03-21 03:20:37,674 - lava_callback - ERROR - Error processing callback for node 69bb6b72a1e2bfb1b3347659: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35534 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35986 - "GET / HTTP/1.0" 200 OK 2026-03-21 03:21:57,636 - lava_callback - INFO - Received callback for node 69bb6bc4a1e2bfb1b3347664 INFO: 172.21.0.1:35992 - "POST /node/69bb6bc4a1e2bfb1b3347664 HTTP/1.0" 202 Accepted 2026-03-21 03:21:57,641 - lava_callback - INFO - Starting async job submit for node 69bb6bc4a1e2bfb1b3347664 2026-03-21 03:21:57,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:21:57,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb6bc4a1e2bfb1b3347664 HTTP/1.1" 200 None 2026-03-21 03:21:57,659 - lava_callback - DEBUG - Processing node 69bb6bc4a1e2bfb1b3347664: baseline-arm64-clabbe 2026-03-21 03:21:57,659 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb6bc4a1e2bfb1b3347664 2026-03-21 03:21:57,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:21:57,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb6bc4a1e2bfb1b3347664 HTTP/1.1" 200 None 2026-03-21 03:21:57,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:21:57,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-21 03:21:57,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:21:57,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb6bc4a1e2bfb1b3347664 HTTP/1.1" 200 None 2026-03-21 03:21:57,711 - lava_callback - ERROR - Error processing callback for node 69bb6bc4a1e2bfb1b3347664: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47212 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40152 - "GET / HTTP/1.0" 200 OK 2026-03-21 03:23:05,674 - lava_callback - INFO - Received callback for node 69bb6c0ea1e2bfb1b334766c INFO: 172.21.0.1:38584 - "POST /node/69bb6c0ea1e2bfb1b334766c HTTP/1.0" 202 Accepted 2026-03-21 03:23:05,680 - lava_callback - INFO - Starting async job submit for node 69bb6c0ea1e2bfb1b334766c 2026-03-21 03:23:05,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:23:05,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb6c0ea1e2bfb1b334766c HTTP/1.1" 200 None 2026-03-21 03:23:05,696 - lava_callback - DEBUG - Processing node 69bb6c0ea1e2bfb1b334766c: baseline-arm64-baylibre 2026-03-21 03:23:05,696 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb6c0ea1e2bfb1b334766c 2026-03-21 03:23:05,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:23:05,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb6c0ea1e2bfb1b334766c HTTP/1.1" 200 None 2026-03-21 03:23:05,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:23:05,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-21 03:23:05,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:23:05,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb6c0ea1e2bfb1b334766c HTTP/1.1" 200 None 2026-03-21 03:23:05,774 - lava_callback - ERROR - Error processing callback for node 69bb6c0ea1e2bfb1b334766c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33106 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49630 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45046 - "GET / HTTP/1.0" 200 OK 2026-03-21 03:24:37,727 - lava_callback - INFO - Received callback for node 69bb6c67a1e2bfb1b3347679 INFO: 172.21.0.1:45054 - "POST /node/69bb6c67a1e2bfb1b3347679 HTTP/1.0" 202 Accepted 2026-03-21 03:24:37,733 - lava_callback - INFO - Starting async job submit for node 69bb6c67a1e2bfb1b3347679 2026-03-21 03:24:37,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:24:37,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb6c67a1e2bfb1b3347679 HTTP/1.1" 200 None 2026-03-21 03:24:37,749 - lava_callback - DEBUG - Processing node 69bb6c67a1e2bfb1b3347679: baseline-arm64-clabbe 2026-03-21 03:24:37,749 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb6c67a1e2bfb1b3347679 2026-03-21 03:24:37,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:24:37,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb6c67a1e2bfb1b3347679 HTTP/1.1" 200 None 2026-03-21 03:24:37,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:24:37,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-21 03:24:37,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:24:37,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb6c67a1e2bfb1b3347679 HTTP/1.1" 200 None 2026-03-21 03:24:37,872 - lava_callback - ERROR - Error processing callback for node 69bb6c67a1e2bfb1b3347679: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36556 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52296 - "GET / HTTP/1.0" 200 OK 2026-03-21 03:25:57,766 - lava_callback - INFO - Received callback for node 69bb6cbaa1e2bfb1b3347685 INFO: 172.21.0.1:52308 - "POST /node/69bb6cbaa1e2bfb1b3347685 HTTP/1.0" 202 Accepted 2026-03-21 03:25:57,771 - lava_callback - INFO - Starting async job submit for node 69bb6cbaa1e2bfb1b3347685 2026-03-21 03:25:57,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:25:57,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb6cbaa1e2bfb1b3347685 HTTP/1.1" 200 None 2026-03-21 03:25:57,788 - lava_callback - DEBUG - Processing node 69bb6cbaa1e2bfb1b3347685: baseline-arm64-clabbe 2026-03-21 03:25:57,788 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb6cbaa1e2bfb1b3347685 2026-03-21 03:25:57,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:25:57,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb6cbaa1e2bfb1b3347685 HTTP/1.1" 200 None 2026-03-21 03:25:57,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:25:57,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-21 03:25:57,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:25:57,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb6cbaa1e2bfb1b3347685 HTTP/1.1" 200 None 2026-03-21 03:25:57,902 - lava_callback - ERROR - Error processing callback for node 69bb6cbaa1e2bfb1b3347685: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43784 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54394 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54396 - "GET / HTTP/1.0" 200 OK 2026-03-21 03:27:17,913 - lava_callback - INFO - Received callback for node 69bb6d0da1e2bfb1b3347693 INFO: 172.21.0.1:54408 - "POST /node/69bb6d0da1e2bfb1b3347693 HTTP/1.0" 202 Accepted 2026-03-21 03:27:17,918 - lava_callback - INFO - Starting async job submit for node 69bb6d0da1e2bfb1b3347693 2026-03-21 03:27:17,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:27:17,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb6d0da1e2bfb1b3347693 HTTP/1.1" 200 None 2026-03-21 03:27:17,935 - lava_callback - DEBUG - Processing node 69bb6d0da1e2bfb1b3347693: baseline-arm64-clabbe 2026-03-21 03:27:17,935 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb6d0da1e2bfb1b3347693 2026-03-21 03:27:17,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:27:17,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb6d0da1e2bfb1b3347693 HTTP/1.1" 200 None 2026-03-21 03:27:17,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:27:17,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-21 03:27:17,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:27:18,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb6d0da1e2bfb1b3347693 HTTP/1.1" 200 None 2026-03-21 03:27:18,014 - lava_callback - ERROR - Error processing callback for node 69bb6d0da1e2bfb1b3347693: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52170 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39256 - "GET / HTTP/1.0" 200 OK 2026-03-21 03:28:45,794 - lava_callback - INFO - Received callback for node 69bb6d59a1e2bfb1b3347698 INFO: 172.21.0.1:54896 - "POST /node/69bb6d59a1e2bfb1b3347698 HTTP/1.0" 202 Accepted 2026-03-21 03:28:45,799 - lava_callback - INFO - Starting async job submit for node 69bb6d59a1e2bfb1b3347698 2026-03-21 03:28:45,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:28:45,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb6d59a1e2bfb1b3347698 HTTP/1.1" 200 None 2026-03-21 03:28:45,816 - lava_callback - DEBUG - Processing node 69bb6d59a1e2bfb1b3347698: baseline-arm64-baylibre 2026-03-21 03:28:45,817 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb6d59a1e2bfb1b3347698 2026-03-21 03:28:45,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:28:45,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb6d59a1e2bfb1b3347698 HTTP/1.1" 200 None 2026-03-21 03:28:45,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:28:45,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-21 03:28:45,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:28:45,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb6d59a1e2bfb1b3347698 HTTP/1.1" 200 None 2026-03-21 03:28:45,888 - lava_callback - ERROR - Error processing callback for node 69bb6d59a1e2bfb1b3347698: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53162 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51074 - "GET / HTTP/1.0" 200 OK 2026-03-21 03:29:57,944 - lava_callback - INFO - Received callback for node 69bb6db1a1e2bfb1b33476a5 INFO: 172.21.0.1:51080 - "POST /node/69bb6db1a1e2bfb1b33476a5 HTTP/1.0" 202 Accepted 2026-03-21 03:29:57,949 - lava_callback - INFO - Starting async job submit for node 69bb6db1a1e2bfb1b33476a5 2026-03-21 03:29:57,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:29:57,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb6db1a1e2bfb1b33476a5 HTTP/1.1" 200 None 2026-03-21 03:29:57,966 - lava_callback - DEBUG - Processing node 69bb6db1a1e2bfb1b33476a5: baseline-arm64-clabbe 2026-03-21 03:29:57,967 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb6db1a1e2bfb1b33476a5 2026-03-21 03:29:57,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:29:57,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb6db1a1e2bfb1b33476a5 HTTP/1.1" 200 None 2026-03-21 03:29:57,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:29:58,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-21 03:29:58,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:29:58,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb6db1a1e2bfb1b33476a5 HTTP/1.1" 200 None 2026-03-21 03:29:58,040 - lava_callback - ERROR - Error processing callback for node 69bb6db1a1e2bfb1b33476a5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49106 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39154 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60608 - "GET / HTTP/1.0" 200 OK 2026-03-21 03:31:37,938 - lava_callback - INFO - Received callback for node 69bb6e04a1e2bfb1b33476b3 INFO: 172.21.0.1:60624 - "POST /node/69bb6e04a1e2bfb1b33476b3 HTTP/1.0" 202 Accepted 2026-03-21 03:31:37,943 - lava_callback - INFO - Starting async job submit for node 69bb6e04a1e2bfb1b33476b3 2026-03-21 03:31:37,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:31:37,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb6e04a1e2bfb1b33476b3 HTTP/1.1" 200 None 2026-03-21 03:31:37,960 - lava_callback - DEBUG - Processing node 69bb6e04a1e2bfb1b33476b3: baseline-arm64-clabbe 2026-03-21 03:31:37,960 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb6e04a1e2bfb1b33476b3 2026-03-21 03:31:37,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:31:37,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb6e04a1e2bfb1b33476b3 HTTP/1.1" 200 None 2026-03-21 03:31:37,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:31:37,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-21 03:31:37,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:31:38,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb6e04a1e2bfb1b33476b3 HTTP/1.1" 200 None 2026-03-21 03:31:38,017 - lava_callback - ERROR - Error processing callback for node 69bb6e04a1e2bfb1b33476b3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50924 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43612 - "GET / HTTP/1.0" 200 OK 2026-03-21 03:32:57,989 - lava_callback - INFO - Received callback for node 69bb6e56a1e2bfb1b33476bd INFO: 172.21.0.1:43620 - "POST /node/69bb6e56a1e2bfb1b33476bd HTTP/1.0" 202 Accepted 2026-03-21 03:32:57,995 - lava_callback - INFO - Starting async job submit for node 69bb6e56a1e2bfb1b33476bd 2026-03-21 03:32:57,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:32:58,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb6e56a1e2bfb1b33476bd HTTP/1.1" 200 None 2026-03-21 03:32:58,012 - lava_callback - DEBUG - Processing node 69bb6e56a1e2bfb1b33476bd: baseline-arm64-clabbe 2026-03-21 03:32:58,012 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb6e56a1e2bfb1b33476bd 2026-03-21 03:32:58,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:32:58,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb6e56a1e2bfb1b33476bd HTTP/1.1" 200 None 2026-03-21 03:32:58,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:32:58,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-21 03:32:58,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:32:58,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb6e56a1e2bfb1b33476bd HTTP/1.1" 200 None 2026-03-21 03:32:58,066 - lava_callback - ERROR - Error processing callback for node 69bb6e56a1e2bfb1b33476bd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52022 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43644 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43658 - "GET / HTTP/1.0" 200 OK 2026-03-21 03:34:18,098 - lava_callback - INFO - Received callback for node 69bb6ea9a1e2bfb1b33476c4 INFO: 172.21.0.1:43674 - "POST /node/69bb6ea9a1e2bfb1b33476c4 HTTP/1.0" 202 Accepted 2026-03-21 03:34:18,104 - lava_callback - INFO - Starting async job submit for node 69bb6ea9a1e2bfb1b33476c4 2026-03-21 03:34:18,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:34:18,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb6ea9a1e2bfb1b33476c4 HTTP/1.1" 200 None 2026-03-21 03:34:18,120 - lava_callback - DEBUG - Processing node 69bb6ea9a1e2bfb1b33476c4: baseline-arm64-clabbe 2026-03-21 03:34:18,120 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb6ea9a1e2bfb1b33476c4 2026-03-21 03:34:18,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:34:18,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb6ea9a1e2bfb1b33476c4 HTTP/1.1" 200 None 2026-03-21 03:34:18,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:34:18,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-21 03:34:18,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:34:18,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb6ea9a1e2bfb1b33476c4 HTTP/1.1" 200 None 2026-03-21 03:34:18,173 - lava_callback - ERROR - Error processing callback for node 69bb6ea9a1e2bfb1b33476c4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54080 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50492 - "GET / HTTP/1.0" 200 OK 2026-03-21 03:35:38,138 - lava_callback - INFO - Received callback for node 69bb6efca1e2bfb1b33476d0 INFO: 172.21.0.1:50506 - "POST /node/69bb6efca1e2bfb1b33476d0 HTTP/1.0" 202 Accepted 2026-03-21 03:35:38,143 - lava_callback - INFO - Starting async job submit for node 69bb6efca1e2bfb1b33476d0 2026-03-21 03:35:38,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:35:38,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb6efca1e2bfb1b33476d0 HTTP/1.1" 200 None 2026-03-21 03:35:38,170 - lava_callback - DEBUG - Processing node 69bb6efca1e2bfb1b33476d0: baseline-arm64-clabbe 2026-03-21 03:35:38,170 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb6efca1e2bfb1b33476d0 2026-03-21 03:35:38,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:35:38,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb6efca1e2bfb1b33476d0 HTTP/1.1" 200 None 2026-03-21 03:35:38,196 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:35:38,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-21 03:35:38,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:35:38,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb6efca1e2bfb1b33476d0 HTTP/1.1" 200 None 2026-03-21 03:35:38,274 - lava_callback - ERROR - Error processing callback for node 69bb6efca1e2bfb1b33476d0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43754 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59694 - "GET / HTTP/1.0" 200 OK 2026-03-21 03:36:58,158 - lava_callback - INFO - Received callback for node 69bb6f4ea1e2bfb1b33476db INFO: 172.21.0.1:59704 - "POST /node/69bb6f4ea1e2bfb1b33476db HTTP/1.0" 202 Accepted 2026-03-21 03:36:58,164 - lava_callback - INFO - Starting async job submit for node 69bb6f4ea1e2bfb1b33476db 2026-03-21 03:36:58,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:36:58,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb6f4ea1e2bfb1b33476db HTTP/1.1" 200 None 2026-03-21 03:36:58,179 - lava_callback - DEBUG - Processing node 69bb6f4ea1e2bfb1b33476db: baseline-arm64-clabbe 2026-03-21 03:36:58,179 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb6f4ea1e2bfb1b33476db 2026-03-21 03:36:58,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:36:58,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb6f4ea1e2bfb1b33476db HTTP/1.1" 200 None 2026-03-21 03:36:58,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:36:58,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-21 03:36:58,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:36:58,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb6f4ea1e2bfb1b33476db HTTP/1.1" 200 None 2026-03-21 03:36:58,238 - lava_callback - ERROR - Error processing callback for node 69bb6f4ea1e2bfb1b33476db: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53428 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34140 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34148 - "GET / HTTP/1.0" 200 OK 2026-03-21 03:38:18,240 - lava_callback - INFO - Received callback for node 69bb6f9fa1e2bfb1b33476e5 INFO: 172.21.0.1:34150 - "POST /node/69bb6f9fa1e2bfb1b33476e5 HTTP/1.0" 202 Accepted 2026-03-21 03:38:18,245 - lava_callback - INFO - Starting async job submit for node 69bb6f9fa1e2bfb1b33476e5 2026-03-21 03:38:18,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:38:18,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb6f9fa1e2bfb1b33476e5 HTTP/1.1" 200 None 2026-03-21 03:38:18,261 - lava_callback - DEBUG - Processing node 69bb6f9fa1e2bfb1b33476e5: baseline-arm64-clabbe 2026-03-21 03:38:18,262 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb6f9fa1e2bfb1b33476e5 2026-03-21 03:38:18,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:38:18,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb6f9fa1e2bfb1b33476e5 HTTP/1.1" 200 None 2026-03-21 03:38:18,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:38:18,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-21 03:38:18,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:38:18,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb6f9fa1e2bfb1b33476e5 HTTP/1.1" 200 None 2026-03-21 03:38:18,328 - lava_callback - ERROR - Error processing callback for node 69bb6f9fa1e2bfb1b33476e5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34938 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41366 - "GET / HTTP/1.0" 200 OK 2026-03-21 03:39:38,356 - lava_callback - INFO - Received callback for node 69bb6ff0a1e2bfb1b33476ed INFO: 172.21.0.1:41368 - "POST /node/69bb6ff0a1e2bfb1b33476ed HTTP/1.0" 202 Accepted 2026-03-21 03:39:38,361 - lava_callback - INFO - Starting async job submit for node 69bb6ff0a1e2bfb1b33476ed 2026-03-21 03:39:38,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:39:38,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb6ff0a1e2bfb1b33476ed HTTP/1.1" 200 None 2026-03-21 03:39:38,389 - lava_callback - DEBUG - Processing node 69bb6ff0a1e2bfb1b33476ed: baseline-arm64-clabbe 2026-03-21 03:39:38,389 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb6ff0a1e2bfb1b33476ed 2026-03-21 03:39:38,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:39:38,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb6ff0a1e2bfb1b33476ed HTTP/1.1" 200 None 2026-03-21 03:39:38,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:39:38,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-21 03:39:38,461 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:39:38,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb6ff0a1e2bfb1b33476ed HTTP/1.1" 200 None 2026-03-21 03:39:38,483 - lava_callback - ERROR - Error processing callback for node 69bb6ff0a1e2bfb1b33476ed: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55862 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42106 - "GET / HTTP/1.0" 200 OK 2026-03-21 03:40:58,422 - lava_callback - INFO - Received callback for node 69bb7044a1e2bfb1b33476f7 INFO: 172.21.0.1:42114 - "POST /node/69bb7044a1e2bfb1b33476f7 HTTP/1.0" 202 Accepted 2026-03-21 03:40:58,428 - lava_callback - INFO - Starting async job submit for node 69bb7044a1e2bfb1b33476f7 2026-03-21 03:40:58,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:40:58,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb7044a1e2bfb1b33476f7 HTTP/1.1" 200 None 2026-03-21 03:40:58,462 - lava_callback - DEBUG - Processing node 69bb7044a1e2bfb1b33476f7: baseline-arm64-clabbe 2026-03-21 03:40:58,463 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb7044a1e2bfb1b33476f7 2026-03-21 03:40:58,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:40:58,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb7044a1e2bfb1b33476f7 HTTP/1.1" 200 None 2026-03-21 03:40:58,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:40:58,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-21 03:40:58,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:40:58,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb7044a1e2bfb1b33476f7 HTTP/1.1" 200 None 2026-03-21 03:40:58,533 - lava_callback - ERROR - Error processing callback for node 69bb7044a1e2bfb1b33476f7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41004 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42926 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57454 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35836 - "GET / HTTP/1.0" 200 OK 2026-03-21 03:43:38,502 - lava_callback - INFO - Received callback for node 69bb70d4a1e2bfb1b3347705 INFO: 172.21.0.1:35842 - "POST /node/69bb70d4a1e2bfb1b3347705 HTTP/1.0" 202 Accepted 2026-03-21 03:43:38,507 - lava_callback - INFO - Starting async job submit for node 69bb70d4a1e2bfb1b3347705 2026-03-21 03:43:38,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:43:38,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb70d4a1e2bfb1b3347705 HTTP/1.1" 200 None 2026-03-21 03:43:38,524 - lava_callback - DEBUG - Processing node 69bb70d4a1e2bfb1b3347705: baseline-arm64-clabbe 2026-03-21 03:43:38,524 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb70d4a1e2bfb1b3347705 2026-03-21 03:43:38,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:43:38,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb70d4a1e2bfb1b3347705 HTTP/1.1" 200 None 2026-03-21 03:43:38,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:43:38,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-21 03:43:38,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:43:38,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb70d4a1e2bfb1b3347705 HTTP/1.1" 200 None 2026-03-21 03:43:38,576 - lava_callback - ERROR - Error processing callback for node 69bb70d4a1e2bfb1b3347705: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37348 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40354 - "GET / HTTP/1.0" 200 OK 2026-03-21 03:44:58,523 - lava_callback - INFO - Received callback for node 69bb7126a1e2bfb1b334770d INFO: 172.21.0.1:40370 - "POST /node/69bb7126a1e2bfb1b334770d HTTP/1.0" 202 Accepted 2026-03-21 03:44:58,529 - lava_callback - INFO - Starting async job submit for node 69bb7126a1e2bfb1b334770d 2026-03-21 03:44:58,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:44:58,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb7126a1e2bfb1b334770d HTTP/1.1" 200 None 2026-03-21 03:44:58,548 - lava_callback - DEBUG - Processing node 69bb7126a1e2bfb1b334770d: baseline-arm64-clabbe 2026-03-21 03:44:58,548 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb7126a1e2bfb1b334770d 2026-03-21 03:44:58,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:44:58,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb7126a1e2bfb1b334770d HTTP/1.1" 200 None 2026-03-21 03:44:58,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:44:58,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-21 03:44:58,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 03:44:58,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb7126a1e2bfb1b334770d HTTP/1.1" 200 None 2026-03-21 03:44:58,632 - lava_callback - ERROR - Error processing callback for node 69bb7126a1e2bfb1b334770d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58000 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42742 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36422 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35542 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53718 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55592 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40912 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43334 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55330 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49724 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55420 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53974 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51720 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54528 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51658 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44350 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38824 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39842 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49022 - "GET / HTTP/1.0" 200 OK 2026-03-21 04:03:07,772 - lava_callback - INFO - Received callback for node 69bb7579a1e2bfb1b3347764 INFO: 172.21.0.1:49032 - "POST /node/69bb7579a1e2bfb1b3347764 HTTP/1.0" 202 Accepted 2026-03-21 04:03:07,777 - lava_callback - INFO - Starting async job submit for node 69bb7579a1e2bfb1b3347764 2026-03-21 04:03:07,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:03:07,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb7579a1e2bfb1b3347764 HTTP/1.1" 200 None 2026-03-21 04:03:07,793 - lava_callback - DEBUG - Processing node 69bb7579a1e2bfb1b3347764: baseline-arm64-baylibre 2026-03-21 04:03:07,793 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb7579a1e2bfb1b3347764 2026-03-21 04:03:07,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:03:07,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb7579a1e2bfb1b3347764 HTTP/1.1" 200 None 2026-03-21 04:03:07,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:03:07,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f7e537ae4d3c8236e55 HTTP/1.1" 200 None 2026-03-21 04:03:07,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:03:07,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb7579a1e2bfb1b3347764 HTTP/1.1" 200 None 2026-03-21 04:03:07,845 - lava_callback - ERROR - Error processing callback for node 69bb7579a1e2bfb1b3347764: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43648 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38650 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.21.0.1:38656 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40826 - "GET / HTTP/1.0" 200 OK 2026-03-21 04:04:27,691 - lava_callback - INFO - Received callback for node 69bb75c9a1e2bfb1b3347771 INFO: 172.21.0.1:40836 - "POST /node/69bb75c9a1e2bfb1b3347771 HTTP/1.0" 202 Accepted 2026-03-21 04:04:27,696 - lava_callback - INFO - Starting async job submit for node 69bb75c9a1e2bfb1b3347771 2026-03-21 04:04:27,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:04:27,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb75c9a1e2bfb1b3347771 HTTP/1.1" 200 None 2026-03-21 04:04:27,712 - lava_callback - DEBUG - Processing node 69bb75c9a1e2bfb1b3347771: baseline-arm64-baylibre 2026-03-21 04:04:27,713 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb75c9a1e2bfb1b3347771 2026-03-21 04:04:27,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:04:27,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb75c9a1e2bfb1b3347771 HTTP/1.1" 200 None 2026-03-21 04:04:27,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:04:27,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f7e537ae4d3c8236e55 HTTP/1.1" 200 None 2026-03-21 04:04:27,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:04:27,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb75c9a1e2bfb1b3347771 HTTP/1.1" 200 None 2026-03-21 04:04:27,813 - lava_callback - ERROR - Error processing callback for node 69bb75c9a1e2bfb1b3347771: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51944 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38254 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40992 - "GET / HTTP/1.0" 200 OK 2026-03-21 04:06:27,886 - lava_callback - INFO - Received callback for node 69bb762fa1e2bfb1b334777f INFO: 172.21.0.1:41006 - "POST /node/69bb762fa1e2bfb1b334777f HTTP/1.0" 202 Accepted 2026-03-21 04:06:27,891 - lava_callback - INFO - Starting async job submit for node 69bb762fa1e2bfb1b334777f 2026-03-21 04:06:27,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:06:27,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb762fa1e2bfb1b334777f HTTP/1.1" 200 None 2026-03-21 04:06:27,920 - lava_callback - DEBUG - Processing node 69bb762fa1e2bfb1b334777f: baseline-arm64-baylibre 2026-03-21 04:06:27,920 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb762fa1e2bfb1b334777f 2026-03-21 04:06:27,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:06:27,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb762fa1e2bfb1b334777f HTTP/1.1" 200 None 2026-03-21 04:06:27,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:06:27,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f7e537ae4d3c8236e55 HTTP/1.1" 200 None 2026-03-21 04:06:27,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:06:28,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb762fa1e2bfb1b334777f HTTP/1.1" 200 None 2026-03-21 04:06:28,012 - lava_callback - ERROR - Error processing callback for node 69bb762fa1e2bfb1b334777f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45552 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34036 - "GET / HTTP/1.0" 200 OK 2026-03-21 04:07:47,930 - lava_callback - INFO - Received callback for node 69bb7680a1e2bfb1b334778b INFO: 172.21.0.1:34040 - "POST /node/69bb7680a1e2bfb1b334778b HTTP/1.0" 202 Accepted 2026-03-21 04:07:47,936 - lava_callback - INFO - Starting async job submit for node 69bb7680a1e2bfb1b334778b 2026-03-21 04:07:47,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:07:47,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb7680a1e2bfb1b334778b HTTP/1.1" 200 None 2026-03-21 04:07:47,965 - lava_callback - DEBUG - Processing node 69bb7680a1e2bfb1b334778b: baseline-arm64-baylibre 2026-03-21 04:07:47,965 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb7680a1e2bfb1b334778b 2026-03-21 04:07:47,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:07:47,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb7680a1e2bfb1b334778b HTTP/1.1" 200 None 2026-03-21 04:07:47,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:07:48,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f7e537ae4d3c8236e55 HTTP/1.1" 200 None 2026-03-21 04:07:48,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:07:48,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb7680a1e2bfb1b334778b HTTP/1.1" 200 None 2026-03-21 04:07:48,030 - lava_callback - ERROR - Error processing callback for node 69bb7680a1e2bfb1b334778b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38076 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43362 - "GET / HTTP/1.0" 200 OK 2026-03-21 04:09:07,978 - lava_callback - INFO - Received callback for node 69bb76d1a1e2bfb1b3347799 INFO: 172.21.0.1:43370 - "POST /node/69bb76d1a1e2bfb1b3347799 HTTP/1.0" 202 Accepted 2026-03-21 04:09:07,983 - lava_callback - INFO - Starting async job submit for node 69bb76d1a1e2bfb1b3347799 2026-03-21 04:09:07,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:09:07,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb76d1a1e2bfb1b3347799 HTTP/1.1" 200 None 2026-03-21 04:09:07,999 - lava_callback - DEBUG - Processing node 69bb76d1a1e2bfb1b3347799: baseline-arm64-baylibre 2026-03-21 04:09:07,999 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb76d1a1e2bfb1b3347799 2026-03-21 04:09:08,000 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:09:08,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb76d1a1e2bfb1b3347799 HTTP/1.1" 200 None 2026-03-21 04:09:08,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:09:08,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f7e537ae4d3c8236e55 HTTP/1.1" 200 None 2026-03-21 04:09:08,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:09:08,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb76d1a1e2bfb1b3347799 HTTP/1.1" 200 None 2026-03-21 04:09:08,083 - lava_callback - ERROR - Error processing callback for node 69bb76d1a1e2bfb1b3347799: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40946 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43746 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48930 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50680 - "GET / HTTP/1.0" 200 OK 2026-03-21 04:11:28,128 - lava_callback - INFO - Received callback for node 69bb775da1e2bfb1b33477ab INFO: 172.21.0.1:50690 - "POST /node/69bb775da1e2bfb1b33477ab HTTP/1.0" 202 Accepted 2026-03-21 04:11:28,133 - lava_callback - INFO - Starting async job submit for node 69bb775da1e2bfb1b33477ab 2026-03-21 04:11:28,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:11:28,149 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb775da1e2bfb1b33477ab HTTP/1.1" 200 None 2026-03-21 04:11:28,150 - lava_callback - DEBUG - Processing node 69bb775da1e2bfb1b33477ab: baseline-arm64-baylibre 2026-03-21 04:11:28,150 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb775da1e2bfb1b33477ab 2026-03-21 04:11:28,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:11:28,164 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb775da1e2bfb1b33477ab HTTP/1.1" 200 None 2026-03-21 04:11:28,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:11:28,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f7e537ae4d3c8236e55 HTTP/1.1" 200 None 2026-03-21 04:11:28,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:11:28,210 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb775da1e2bfb1b33477ab HTTP/1.1" 200 None 2026-03-21 04:11:28,212 - lava_callback - ERROR - Error processing callback for node 69bb775da1e2bfb1b33477ab: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35656 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41246 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43592 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50928 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45248 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41514 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51130 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37802 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49836 - "GET / HTTP/1.0" 200 OK 2026-03-21 04:19:39,917 - lava_callback - INFO - Received callback for node 69bb7945a1e2bfb1b33477cd INFO: 172.21.0.1:49982 - "POST /node/69bb7945a1e2bfb1b33477cd HTTP/1.0" 202 Accepted 2026-03-21 04:19:39,923 - lava_callback - INFO - Starting async job submit for node 69bb7945a1e2bfb1b33477cd 2026-03-21 04:19:39,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:19:39,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb7945a1e2bfb1b33477cd HTTP/1.1" 200 None 2026-03-21 04:19:39,939 - lava_callback - DEBUG - Processing node 69bb7945a1e2bfb1b33477cd: baseline-arm64-clabbe 2026-03-21 04:19:39,940 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb7945a1e2bfb1b33477cd 2026-03-21 04:19:39,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:19:39,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb7945a1e2bfb1b33477cd HTTP/1.1" 200 None 2026-03-21 04:19:39,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:19:39,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f7e537ae4d3c8236e55 HTTP/1.1" 200 None 2026-03-21 04:19:39,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:19:40,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb7945a1e2bfb1b33477cd HTTP/1.1" 200 None 2026-03-21 04:19:40,028 - lava_callback - ERROR - Error processing callback for node 69bb7945a1e2bfb1b33477cd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38622 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33612 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56158 - "GET / HTTP/1.0" 200 OK 2026-03-21 04:21:40,060 - lava_callback - INFO - Received callback for node 69bb79bea1e2bfb1b33477db INFO: 172.21.0.1:48472 - "POST /node/69bb79bea1e2bfb1b33477db HTTP/1.0" 202 Accepted 2026-03-21 04:21:40,066 - lava_callback - INFO - Starting async job submit for node 69bb79bea1e2bfb1b33477db 2026-03-21 04:21:40,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:21:40,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb79bea1e2bfb1b33477db HTTP/1.1" 200 None 2026-03-21 04:21:40,084 - lava_callback - DEBUG - Processing node 69bb79bea1e2bfb1b33477db: baseline-arm64-clabbe 2026-03-21 04:21:40,084 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb79bea1e2bfb1b33477db 2026-03-21 04:21:40,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:21:40,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb79bea1e2bfb1b33477db HTTP/1.1" 200 None 2026-03-21 04:21:40,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:21:40,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f7e537ae4d3c8236e55 HTTP/1.1" 200 None 2026-03-21 04:21:40,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:21:40,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb79bea1e2bfb1b33477db HTTP/1.1" 200 None 2026-03-21 04:21:40,177 - lava_callback - ERROR - Error processing callback for node 69bb79bea1e2bfb1b33477db: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60228 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51156 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55110 - "GET / HTTP/1.0" 200 OK 2026-03-21 04:23:40,087 - lava_callback - INFO - Received callback for node 69bb7a36a1e2bfb1b33477e9 INFO: 172.21.0.1:59150 - "POST /node/69bb7a36a1e2bfb1b33477e9 HTTP/1.0" 202 Accepted 2026-03-21 04:23:40,092 - lava_callback - INFO - Starting async job submit for node 69bb7a36a1e2bfb1b33477e9 2026-03-21 04:23:40,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:23:40,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb7a36a1e2bfb1b33477e9 HTTP/1.1" 200 None 2026-03-21 04:23:40,122 - lava_callback - DEBUG - Processing node 69bb7a36a1e2bfb1b33477e9: baseline-arm64-clabbe 2026-03-21 04:23:40,122 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb7a36a1e2bfb1b33477e9 2026-03-21 04:23:40,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:23:40,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb7a36a1e2bfb1b33477e9 HTTP/1.1" 200 None 2026-03-21 04:23:40,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:23:40,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f7e537ae4d3c8236e55 HTTP/1.1" 200 None 2026-03-21 04:23:40,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:23:40,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb7a36a1e2bfb1b33477e9 HTTP/1.1" 200 None 2026-03-21 04:23:40,214 - lava_callback - ERROR - Error processing callback for node 69bb7a36a1e2bfb1b33477e9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35694 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50238 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42400 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54082 - "GET / HTTP/1.0" 200 OK 2026-03-21 04:26:40,220 - lava_callback - INFO - Received callback for node 69bb7aeba1e2bfb1b33477f8 INFO: 172.21.0.1:53560 - "POST /node/69bb7aeba1e2bfb1b33477f8 HTTP/1.0" 202 Accepted 2026-03-21 04:26:40,225 - lava_callback - INFO - Starting async job submit for node 69bb7aeba1e2bfb1b33477f8 2026-03-21 04:26:40,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:26:40,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb7aeba1e2bfb1b33477f8 HTTP/1.1" 200 None 2026-03-21 04:26:40,245 - lava_callback - DEBUG - Processing node 69bb7aeba1e2bfb1b33477f8: baseline-arm64-clabbe 2026-03-21 04:26:40,246 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb7aeba1e2bfb1b33477f8 2026-03-21 04:26:40,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:26:40,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb7aeba1e2bfb1b33477f8 HTTP/1.1" 200 None 2026-03-21 04:26:40,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:26:40,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f7e537ae4d3c8236e55 HTTP/1.1" 200 None 2026-03-21 04:26:40,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:26:40,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb7aeba1e2bfb1b33477f8 HTTP/1.1" 200 None 2026-03-21 04:26:40,326 - lava_callback - ERROR - Error processing callback for node 69bb7aeba1e2bfb1b33477f8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55362 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49184 - "GET / HTTP/1.0" 200 OK 2026-03-21 04:28:00,251 - lava_callback - INFO - Received callback for node 69bb7b3ca1e2bfb1b3347805 INFO: 172.21.0.1:43582 - "POST /node/69bb7b3ca1e2bfb1b3347805 HTTP/1.0" 202 Accepted 2026-03-21 04:28:00,256 - lava_callback - INFO - Starting async job submit for node 69bb7b3ca1e2bfb1b3347805 2026-03-21 04:28:00,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:28:00,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb7b3ca1e2bfb1b3347805 HTTP/1.1" 200 None 2026-03-21 04:28:00,295 - lava_callback - DEBUG - Processing node 69bb7b3ca1e2bfb1b3347805: baseline-arm64-clabbe 2026-03-21 04:28:00,295 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb7b3ca1e2bfb1b3347805 2026-03-21 04:28:00,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:28:00,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb7b3ca1e2bfb1b3347805 HTTP/1.1" 200 None 2026-03-21 04:28:00,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:28:00,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f7e537ae4d3c8236e55 HTTP/1.1" 200 None 2026-03-21 04:28:00,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:28:00,351 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb7b3ca1e2bfb1b3347805 HTTP/1.1" 200 None 2026-03-21 04:28:00,356 - lava_callback - ERROR - Error processing callback for node 69bb7b3ca1e2bfb1b3347805: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37504 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47306 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47318 - "GET / HTTP/1.0" 200 OK 2026-03-21 04:29:20,353 - lava_callback - INFO - Received callback for node 69bb7b8ea1e2bfb1b334780f INFO: 172.21.0.1:41496 - "POST /node/69bb7b8ea1e2bfb1b334780f HTTP/1.0" 202 Accepted 2026-03-21 04:29:20,358 - lava_callback - INFO - Starting async job submit for node 69bb7b8ea1e2bfb1b334780f 2026-03-21 04:29:20,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:29:20,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb7b8ea1e2bfb1b334780f HTTP/1.1" 200 None 2026-03-21 04:29:20,381 - lava_callback - DEBUG - Processing node 69bb7b8ea1e2bfb1b334780f: baseline-arm64-clabbe 2026-03-21 04:29:20,381 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb7b8ea1e2bfb1b334780f 2026-03-21 04:29:20,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:29:20,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb7b8ea1e2bfb1b334780f HTTP/1.1" 200 None 2026-03-21 04:29:20,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:29:20,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f7e537ae4d3c8236e55 HTTP/1.1" 200 None 2026-03-21 04:29:20,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:29:20,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb7b8ea1e2bfb1b334780f HTTP/1.1" 200 None 2026-03-21 04:29:20,495 - lava_callback - ERROR - Error processing callback for node 69bb7b8ea1e2bfb1b334780f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35618 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34392 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54852 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42914 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45852 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56430 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33312 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47832 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43768 - "GET / HTTP/1.0" 200 OK 2026-03-21 04:38:09,691 - lava_callback - INFO - Received callback for node 69bb7da8a1e2bfb1b3347828 INFO: 172.21.0.1:43774 - "POST /node/69bb7da8a1e2bfb1b3347828 HTTP/1.0" 202 Accepted 2026-03-21 04:38:09,696 - lava_callback - INFO - Starting async job submit for node 69bb7da8a1e2bfb1b3347828 2026-03-21 04:38:09,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:38:09,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb7da8a1e2bfb1b3347828 HTTP/1.1" 200 None 2026-03-21 04:38:09,715 - lava_callback - DEBUG - Processing node 69bb7da8a1e2bfb1b3347828: baseline-arm-baylibre 2026-03-21 04:38:09,715 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb7da8a1e2bfb1b3347828 2026-03-21 04:38:09,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:38:09,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb7da8a1e2bfb1b3347828 HTTP/1.1" 200 None 2026-03-21 04:38:09,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:38:09,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-21 04:38:09,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:38:09,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb7da8a1e2bfb1b3347828 HTTP/1.1" 200 None 2026-03-21 04:38:09,836 - lava_callback - ERROR - Error processing callback for node 69bb7da8a1e2bfb1b3347828: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51466 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59414 - "GET / HTTP/1.0" 200 OK 2026-03-21 04:39:09,696 - lava_callback - INFO - Received callback for node 69bb7ddea1e2bfb1b3347834 INFO: 172.21.0.1:59424 - "POST /node/69bb7ddea1e2bfb1b3347834 HTTP/1.0" 202 Accepted 2026-03-21 04:39:09,701 - lava_callback - INFO - Starting async job submit for node 69bb7ddea1e2bfb1b3347834 2026-03-21 04:39:09,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:39:09,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb7ddea1e2bfb1b3347834 HTTP/1.1" 200 None 2026-03-21 04:39:09,736 - lava_callback - DEBUG - Processing node 69bb7ddea1e2bfb1b3347834: baseline-arm-baylibre 2026-03-21 04:39:09,737 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb7ddea1e2bfb1b3347834 2026-03-21 04:39:09,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:39:09,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb7ddea1e2bfb1b3347834 HTTP/1.1" 200 None 2026-03-21 04:39:09,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:39:09,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-21 04:39:09,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:39:09,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb7ddea1e2bfb1b3347834 HTTP/1.1" 200 None 2026-03-21 04:39:09,793 - lava_callback - ERROR - Error processing callback for node 69bb7ddea1e2bfb1b3347834: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37006 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58064 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33418 - "GET / HTTP/1.0" 200 OK 2026-03-21 04:40:49,784 - lava_callback - INFO - Received callback for node 69bb7e4ea1e2bfb1b334785c INFO: 172.21.0.1:38828 - "POST /node/69bb7e4ea1e2bfb1b334785c HTTP/1.0" 202 Accepted 2026-03-21 04:40:49,790 - lava_callback - INFO - Starting async job submit for node 69bb7e4ea1e2bfb1b334785c 2026-03-21 04:40:49,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:40:49,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb7e4ea1e2bfb1b334785c HTTP/1.1" 200 None 2026-03-21 04:40:49,806 - lava_callback - DEBUG - Processing node 69bb7e4ea1e2bfb1b334785c: baseline-arm-baylibre 2026-03-21 04:40:49,806 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb7e4ea1e2bfb1b334785c 2026-03-21 04:40:49,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:40:49,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb7e4ea1e2bfb1b334785c HTTP/1.1" 200 None 2026-03-21 04:40:49,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:40:49,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-21 04:40:49,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:40:49,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb7e4ea1e2bfb1b334785c HTTP/1.1" 200 None 2026-03-21 04:40:49,886 - lava_callback - ERROR - Error processing callback for node 69bb7e4ea1e2bfb1b334785c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48768 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34626 - "GET / HTTP/1.0" 200 OK 2026-03-21 04:41:29,806 - lava_callback - INFO - Received callback for node 69bb7e72a1e2bfb1b334786d INFO: 172.21.0.1:54368 - "POST /node/69bb7e72a1e2bfb1b334786d HTTP/1.0" 202 Accepted 2026-03-21 04:41:29,811 - lava_callback - INFO - Starting async job submit for node 69bb7e72a1e2bfb1b334786d 2026-03-21 04:41:29,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:41:29,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb7e72a1e2bfb1b334786d HTTP/1.1" 200 None 2026-03-21 04:41:29,828 - lava_callback - DEBUG - Processing node 69bb7e72a1e2bfb1b334786d: baseline-arm-baylibre 2026-03-21 04:41:29,828 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb7e72a1e2bfb1b334786d 2026-03-21 04:41:29,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:41:29,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb7e72a1e2bfb1b334786d HTTP/1.1" 200 None 2026-03-21 04:41:29,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:41:29,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-21 04:41:29,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:41:29,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb7e72a1e2bfb1b334786d HTTP/1.1" 200 None 2026-03-21 04:41:29,913 - lava_callback - ERROR - Error processing callback for node 69bb7e72a1e2bfb1b334786d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46106 - "GET / HTTP/1.0" 200 OK 2026-03-21 04:42:09,832 - lava_callback - INFO - Received callback for node 69bb7e98a1e2bfb1b334787a INFO: 172.21.0.1:54334 - "POST /node/69bb7e98a1e2bfb1b334787a HTTP/1.0" 202 Accepted 2026-03-21 04:42:09,838 - lava_callback - INFO - Starting async job submit for node 69bb7e98a1e2bfb1b334787a 2026-03-21 04:42:09,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:42:09,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb7e98a1e2bfb1b334787a HTTP/1.1" 200 None 2026-03-21 04:42:09,856 - lava_callback - DEBUG - Processing node 69bb7e98a1e2bfb1b334787a: baseline-arm-baylibre 2026-03-21 04:42:09,857 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb7e98a1e2bfb1b334787a 2026-03-21 04:42:09,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:42:09,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb7e98a1e2bfb1b334787a HTTP/1.1" 200 None 2026-03-21 04:42:09,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:42:09,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-21 04:42:09,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:42:09,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb7e98a1e2bfb1b334787a HTTP/1.1" 200 None 2026-03-21 04:42:09,918 - lava_callback - ERROR - Error processing callback for node 69bb7e98a1e2bfb1b334787a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54342 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49952 - "GET / HTTP/1.0" 200 OK 2026-03-21 04:42:49,837 - lava_callback - INFO - Received callback for node 69bb7ebca1e2bfb1b3347884 INFO: 172.21.0.1:60956 - "POST /node/69bb7ebca1e2bfb1b3347884 HTTP/1.0" 202 Accepted 2026-03-21 04:42:49,842 - lava_callback - INFO - Starting async job submit for node 69bb7ebca1e2bfb1b3347884 2026-03-21 04:42:49,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:42:49,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb7ebca1e2bfb1b3347884 HTTP/1.1" 200 None 2026-03-21 04:42:49,860 - lava_callback - DEBUG - Processing node 69bb7ebca1e2bfb1b3347884: baseline-arm-baylibre 2026-03-21 04:42:49,860 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb7ebca1e2bfb1b3347884 2026-03-21 04:42:49,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:42:49,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb7ebca1e2bfb1b3347884 HTTP/1.1" 200 None 2026-03-21 04:42:49,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:42:49,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-21 04:42:49,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:42:49,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb7ebca1e2bfb1b3347884 HTTP/1.1" 200 None 2026-03-21 04:42:49,922 - lava_callback - ERROR - Error processing callback for node 69bb7ebca1e2bfb1b3347884: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48696 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38866 - "GET / HTTP/1.0" 200 OK 2026-03-21 04:43:29,998 - lava_callback - INFO - Received callback for node 69bb7edfa1e2bfb1b3347896 INFO: 172.21.0.1:38274 - "POST /node/69bb7edfa1e2bfb1b3347896 HTTP/1.0" 202 Accepted 2026-03-21 04:43:30,004 - lava_callback - INFO - Starting async job submit for node 69bb7edfa1e2bfb1b3347896 2026-03-21 04:43:30,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:43:30,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb7edfa1e2bfb1b3347896 HTTP/1.1" 200 None 2026-03-21 04:43:30,020 - lava_callback - DEBUG - Processing node 69bb7edfa1e2bfb1b3347896: baseline-arm-baylibre 2026-03-21 04:43:30,020 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb7edfa1e2bfb1b3347896 2026-03-21 04:43:30,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:43:30,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb7edfa1e2bfb1b3347896 HTTP/1.1" 200 None 2026-03-21 04:43:30,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:43:30,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-21 04:43:30,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:43:30,110 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb7edfa1e2bfb1b3347896 HTTP/1.1" 200 None 2026-03-21 04:43:30,111 - lava_callback - ERROR - Error processing callback for node 69bb7edfa1e2bfb1b3347896: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38140 - "GET / HTTP/1.0" 200 OK 2026-03-21 04:43:49,890 - lava_callback - INFO - Received callback for node 69bb7f03a1e2bfb1b334789e INFO: 172.21.0.1:54366 - "POST /node/69bb7f03a1e2bfb1b334789e HTTP/1.0" 202 Accepted 2026-03-21 04:43:49,895 - lava_callback - INFO - Starting async job submit for node 69bb7f03a1e2bfb1b334789e 2026-03-21 04:43:49,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:43:49,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb7f03a1e2bfb1b334789e HTTP/1.1" 200 None 2026-03-21 04:43:49,911 - lava_callback - DEBUG - Processing node 69bb7f03a1e2bfb1b334789e: baseline-arm-baylibre 2026-03-21 04:43:49,912 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb7f03a1e2bfb1b334789e 2026-03-21 04:43:49,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:43:49,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb7f03a1e2bfb1b334789e HTTP/1.1" 200 None 2026-03-21 04:43:49,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:43:49,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-21 04:43:49,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:43:49,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb7f03a1e2bfb1b334789e HTTP/1.1" 200 None 2026-03-21 04:43:49,975 - lava_callback - ERROR - Error processing callback for node 69bb7f03a1e2bfb1b334789e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37478 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43002 - "GET / HTTP/1.0" 200 OK 2026-03-21 04:44:29,946 - lava_callback - INFO - Received callback for node 69bb7f28a1e2bfb1b33478aa INFO: 172.21.0.1:40366 - "POST /node/69bb7f28a1e2bfb1b33478aa HTTP/1.0" 202 Accepted 2026-03-21 04:44:29,952 - lava_callback - INFO - Starting async job submit for node 69bb7f28a1e2bfb1b33478aa 2026-03-21 04:44:29,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:44:29,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb7f28a1e2bfb1b33478aa HTTP/1.1" 200 None 2026-03-21 04:44:29,968 - lava_callback - DEBUG - Processing node 69bb7f28a1e2bfb1b33478aa: baseline-arm-baylibre 2026-03-21 04:44:29,968 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb7f28a1e2bfb1b33478aa 2026-03-21 04:44:29,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:44:29,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb7f28a1e2bfb1b33478aa HTTP/1.1" 200 None 2026-03-21 04:44:30,000 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:44:30,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-21 04:44:30,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:44:30,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb7f28a1e2bfb1b33478aa HTTP/1.1" 200 None 2026-03-21 04:44:30,055 - lava_callback - ERROR - Error processing callback for node 69bb7f28a1e2bfb1b33478aa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47500 - "GET / HTTP/1.0" 200 OK 2026-03-21 04:45:10,055 - lava_callback - INFO - Received callback for node 69bb7f4aa1e2bfb1b33478b9 INFO: 172.21.0.1:57520 - "POST /node/69bb7f4aa1e2bfb1b33478b9 HTTP/1.0" 202 Accepted 2026-03-21 04:45:10,061 - lava_callback - INFO - Starting async job submit for node 69bb7f4aa1e2bfb1b33478b9 2026-03-21 04:45:10,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:45:10,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb7f4aa1e2bfb1b33478b9 HTTP/1.1" 200 None 2026-03-21 04:45:10,077 - lava_callback - DEBUG - Processing node 69bb7f4aa1e2bfb1b33478b9: baseline-arm-baylibre 2026-03-21 04:45:10,077 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb7f4aa1e2bfb1b33478b9 2026-03-21 04:45:10,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:45:10,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb7f4aa1e2bfb1b33478b9 HTTP/1.1" 200 None 2026-03-21 04:45:10,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:45:10,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-21 04:45:10,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:45:10,130 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb7f4aa1e2bfb1b33478b9 HTTP/1.1" 200 None 2026-03-21 04:45:10,131 - lava_callback - ERROR - Error processing callback for node 69bb7f4aa1e2bfb1b33478b9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57530 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32776 - "GET / HTTP/1.0" 200 OK 2026-03-21 04:45:50,027 - lava_callback - INFO - Received callback for node 69bb7f70a1e2bfb1b33478c6 INFO: 172.21.0.1:42982 - "POST /node/69bb7f70a1e2bfb1b33478c6 HTTP/1.0" 202 Accepted 2026-03-21 04:45:50,032 - lava_callback - INFO - Starting async job submit for node 69bb7f70a1e2bfb1b33478c6 2026-03-21 04:45:50,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:45:50,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb7f70a1e2bfb1b33478c6 HTTP/1.1" 200 None 2026-03-21 04:45:50,051 - lava_callback - DEBUG - Processing node 69bb7f70a1e2bfb1b33478c6: baseline-arm-baylibre 2026-03-21 04:45:50,051 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb7f70a1e2bfb1b33478c6 2026-03-21 04:45:50,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:45:50,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb7f70a1e2bfb1b33478c6 HTTP/1.1" 200 None 2026-03-21 04:45:50,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:45:50,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-21 04:45:50,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:45:50,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb7f70a1e2bfb1b33478c6 HTTP/1.1" 200 None 2026-03-21 04:45:50,117 - lava_callback - ERROR - Error processing callback for node 69bb7f70a1e2bfb1b33478c6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52028 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55882 - "GET / HTTP/1.0" 200 OK 2026-03-21 04:46:30,080 - lava_callback - INFO - Received callback for node 69bb7f91a1e2bfb1b33478d4 INFO: 172.21.0.1:53124 - "POST /node/69bb7f91a1e2bfb1b33478d4 HTTP/1.0" 202 Accepted 2026-03-21 04:46:30,086 - lava_callback - INFO - Starting async job submit for node 69bb7f91a1e2bfb1b33478d4 2026-03-21 04:46:30,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:46:30,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb7f91a1e2bfb1b33478d4 HTTP/1.1" 200 None 2026-03-21 04:46:30,103 - lava_callback - DEBUG - Processing node 69bb7f91a1e2bfb1b33478d4: baseline-arm-baylibre 2026-03-21 04:46:30,103 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb7f91a1e2bfb1b33478d4 2026-03-21 04:46:30,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:46:30,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb7f91a1e2bfb1b33478d4 HTTP/1.1" 200 None 2026-03-21 04:46:30,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:46:30,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-21 04:46:30,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:46:30,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb7f91a1e2bfb1b33478d4 HTTP/1.1" 200 None 2026-03-21 04:46:30,171 - lava_callback - ERROR - Error processing callback for node 69bb7f91a1e2bfb1b33478d4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39726 - "GET / HTTP/1.0" 200 OK 2026-03-21 04:46:50,034 - lava_callback - INFO - Received callback for node 69bb7fb6a1e2bfb1b33478e0 INFO: 172.21.0.1:56938 - "POST /node/69bb7fb6a1e2bfb1b33478e0 HTTP/1.0" 202 Accepted 2026-03-21 04:46:50,039 - lava_callback - INFO - Starting async job submit for node 69bb7fb6a1e2bfb1b33478e0 2026-03-21 04:46:50,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:46:50,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb7fb6a1e2bfb1b33478e0 HTTP/1.1" 200 None 2026-03-21 04:46:50,065 - lava_callback - DEBUG - Processing node 69bb7fb6a1e2bfb1b33478e0: baseline-arm-baylibre 2026-03-21 04:46:50,065 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb7fb6a1e2bfb1b33478e0 2026-03-21 04:46:50,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:46:50,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb7fb6a1e2bfb1b33478e0 HTTP/1.1" 200 None 2026-03-21 04:46:50,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:46:50,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-21 04:46:50,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:46:50,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb7fb6a1e2bfb1b33478e0 HTTP/1.1" 200 None 2026-03-21 04:46:50,132 - lava_callback - ERROR - Error processing callback for node 69bb7fb6a1e2bfb1b33478e0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52462 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55918 - "GET / HTTP/1.0" 200 OK 2026-03-21 04:47:30,109 - lava_callback - INFO - Received callback for node 69bb7fd9a1e2bfb1b33478ee INFO: 172.21.0.1:56922 - "POST /node/69bb7fd9a1e2bfb1b33478ee HTTP/1.0" 202 Accepted 2026-03-21 04:47:30,115 - lava_callback - INFO - Starting async job submit for node 69bb7fd9a1e2bfb1b33478ee 2026-03-21 04:47:30,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:47:30,130 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb7fd9a1e2bfb1b33478ee HTTP/1.1" 200 None 2026-03-21 04:47:30,131 - lava_callback - DEBUG - Processing node 69bb7fd9a1e2bfb1b33478ee: baseline-arm-baylibre 2026-03-21 04:47:30,131 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb7fd9a1e2bfb1b33478ee 2026-03-21 04:47:30,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:47:30,145 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb7fd9a1e2bfb1b33478ee HTTP/1.1" 200 None 2026-03-21 04:47:30,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:47:30,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-21 04:47:30,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:47:30,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb7fd9a1e2bfb1b33478ee HTTP/1.1" 200 None 2026-03-21 04:47:30,214 - lava_callback - ERROR - Error processing callback for node 69bb7fd9a1e2bfb1b33478ee: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45740 - "GET / HTTP/1.0" 200 OK 2026-03-21 04:48:01,147 - lava_callback - INFO - Received callback for node 69bb7ffda1e2bfb1b33478fc INFO: 172.21.0.1:35254 - "POST /node/69bb7ffda1e2bfb1b33478fc HTTP/1.0" 202 Accepted 2026-03-21 04:48:01,152 - lava_callback - INFO - Starting async job submit for node 69bb7ffda1e2bfb1b33478fc 2026-03-21 04:48:01,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:48:01,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb7ffda1e2bfb1b33478fc HTTP/1.1" 200 None 2026-03-21 04:48:01,168 - lava_callback - DEBUG - Processing node 69bb7ffda1e2bfb1b33478fc: baseline-arm-clabbe 2026-03-21 04:48:01,168 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb7ffda1e2bfb1b33478fc 2026-03-21 04:48:01,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:48:01,183 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb7ffda1e2bfb1b33478fc HTTP/1.1" 200 None 2026-03-21 04:48:01,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:48:01,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-21 04:48:01,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:48:01,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb7ffda1e2bfb1b33478fc HTTP/1.1" 200 None 2026-03-21 04:48:01,229 - lava_callback - ERROR - Error processing callback for node 69bb7ffda1e2bfb1b33478fc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35214 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40686 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44408 - "GET / HTTP/1.0" 200 OK 2026-03-21 04:50:01,155 - lava_callback - INFO - Received callback for node 69bb8067a1e2bfb1b334791f INFO: 172.21.0.1:39044 - "POST /node/69bb8067a1e2bfb1b334791f HTTP/1.0" 202 Accepted 2026-03-21 04:50:01,160 - lava_callback - INFO - Starting async job submit for node 69bb8067a1e2bfb1b334791f 2026-03-21 04:50:01,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:50:01,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb8067a1e2bfb1b334791f HTTP/1.1" 200 None 2026-03-21 04:50:01,185 - lava_callback - DEBUG - Processing node 69bb8067a1e2bfb1b334791f: baseline-arm-clabbe 2026-03-21 04:50:01,186 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb8067a1e2bfb1b334791f 2026-03-21 04:50:01,186 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:50:01,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb8067a1e2bfb1b334791f HTTP/1.1" 200 None 2026-03-21 04:50:01,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:50:01,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-21 04:50:01,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:50:01,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb8067a1e2bfb1b334791f HTTP/1.1" 200 None 2026-03-21 04:50:01,248 - lava_callback - ERROR - Error processing callback for node 69bb8067a1e2bfb1b334791f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50580 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49736 - "GET / HTTP/1.0" 200 OK 2026-03-21 04:50:41,153 - lava_callback - INFO - Received callback for node 69bb808ca1e2bfb1b334792c INFO: 172.21.0.1:60052 - "POST /node/69bb808ca1e2bfb1b334792c HTTP/1.0" 202 Accepted 2026-03-21 04:50:41,158 - lava_callback - INFO - Starting async job submit for node 69bb808ca1e2bfb1b334792c 2026-03-21 04:50:41,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:50:41,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb808ca1e2bfb1b334792c HTTP/1.1" 200 None 2026-03-21 04:50:41,175 - lava_callback - DEBUG - Processing node 69bb808ca1e2bfb1b334792c: baseline-arm-clabbe 2026-03-21 04:50:41,175 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb808ca1e2bfb1b334792c 2026-03-21 04:50:41,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:50:41,190 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb808ca1e2bfb1b334792c HTTP/1.1" 200 None 2026-03-21 04:50:41,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:50:41,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-21 04:50:41,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:50:41,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb808ca1e2bfb1b334792c HTTP/1.1" 200 None 2026-03-21 04:50:41,232 - lava_callback - ERROR - Error processing callback for node 69bb808ca1e2bfb1b334792c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54394 - "GET / HTTP/1.0" 200 OK 2026-03-21 04:51:01,190 - lava_callback - INFO - Received callback for node 69bb80b0a1e2bfb1b3347935 INFO: 172.21.0.1:60998 - "POST /node/69bb80b0a1e2bfb1b3347935 HTTP/1.0" 202 Accepted 2026-03-21 04:51:01,196 - lava_callback - INFO - Starting async job submit for node 69bb80b0a1e2bfb1b3347935 2026-03-21 04:51:01,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:51:01,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb80b0a1e2bfb1b3347935 HTTP/1.1" 200 None 2026-03-21 04:51:01,217 - lava_callback - DEBUG - Processing node 69bb80b0a1e2bfb1b3347935: baseline-arm-clabbe 2026-03-21 04:51:01,217 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb80b0a1e2bfb1b3347935 2026-03-21 04:51:01,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:51:01,231 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb80b0a1e2bfb1b3347935 HTTP/1.1" 200 None 2026-03-21 04:51:01,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:51:01,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-21 04:51:01,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:51:01,288 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb80b0a1e2bfb1b3347935 HTTP/1.1" 200 None 2026-03-21 04:51:01,294 - lava_callback - ERROR - Error processing callback for node 69bb80b0a1e2bfb1b3347935: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:32768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55292 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43798 - "GET / HTTP/1.0" 200 OK 2026-03-21 04:51:41,143 - lava_callback - INFO - Received callback for node 69bb80d5a1e2bfb1b3347941 INFO: 172.21.0.1:45650 - "POST /node/69bb80d5a1e2bfb1b3347941 HTTP/1.0" 202 Accepted 2026-03-21 04:51:41,149 - lava_callback - INFO - Starting async job submit for node 69bb80d5a1e2bfb1b3347941 2026-03-21 04:51:41,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:51:41,164 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb80d5a1e2bfb1b3347941 HTTP/1.1" 200 None 2026-03-21 04:51:41,164 - lava_callback - DEBUG - Processing node 69bb80d5a1e2bfb1b3347941: baseline-arm-clabbe 2026-03-21 04:51:41,165 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb80d5a1e2bfb1b3347941 2026-03-21 04:51:41,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:51:41,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb80d5a1e2bfb1b3347941 HTTP/1.1" 200 None 2026-03-21 04:51:41,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:51:41,193 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-21 04:51:41,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:51:41,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb80d5a1e2bfb1b3347941 HTTP/1.1" 200 None 2026-03-21 04:51:41,216 - lava_callback - ERROR - Error processing callback for node 69bb80d5a1e2bfb1b3347941: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38778 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38790 - "GET / HTTP/1.0" 200 OK 2026-03-21 04:52:21,287 - lava_callback - INFO - Received callback for node 69bb80fba1e2bfb1b3347952 INFO: 172.21.0.1:46660 - "POST /node/69bb80fba1e2bfb1b3347952 HTTP/1.0" 202 Accepted 2026-03-21 04:52:21,293 - lava_callback - INFO - Starting async job submit for node 69bb80fba1e2bfb1b3347952 2026-03-21 04:52:21,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:52:21,310 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb80fba1e2bfb1b3347952 HTTP/1.1" 200 None 2026-03-21 04:52:21,311 - lava_callback - DEBUG - Processing node 69bb80fba1e2bfb1b3347952: baseline-arm-clabbe 2026-03-21 04:52:21,311 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb80fba1e2bfb1b3347952 2026-03-21 04:52:21,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:52:21,325 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb80fba1e2bfb1b3347952 HTTP/1.1" 200 None 2026-03-21 04:52:21,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:52:21,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-21 04:52:21,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:52:21,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb80fba1e2bfb1b3347952 HTTP/1.1" 200 None 2026-03-21 04:52:21,370 - lava_callback - ERROR - Error processing callback for node 69bb80fba1e2bfb1b3347952: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49392 - "GET / HTTP/1.0" 200 OK 2026-03-21 04:53:01,271 - lava_callback - INFO - Received callback for node 69bb8122a1e2bfb1b3347959 INFO: 172.21.0.1:45388 - "POST /node/69bb8122a1e2bfb1b3347959 HTTP/1.0" 202 Accepted 2026-03-21 04:53:01,276 - lava_callback - INFO - Starting async job submit for node 69bb8122a1e2bfb1b3347959 2026-03-21 04:53:01,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:53:01,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb8122a1e2bfb1b3347959 HTTP/1.1" 200 None 2026-03-21 04:53:01,292 - lava_callback - DEBUG - Processing node 69bb8122a1e2bfb1b3347959: baseline-arm-clabbe 2026-03-21 04:53:01,293 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb8122a1e2bfb1b3347959 2026-03-21 04:53:01,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:53:01,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb8122a1e2bfb1b3347959 HTTP/1.1" 200 None 2026-03-21 04:53:01,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:53:01,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-21 04:53:01,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:53:01,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb8122a1e2bfb1b3347959 HTTP/1.1" 200 None 2026-03-21 04:53:01,356 - lava_callback - ERROR - Error processing callback for node 69bb8122a1e2bfb1b3347959: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51690 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51694 - "GET / HTTP/1.0" 200 OK 2026-03-21 04:53:18,781 - lava_callback - INFO - Received callback for node 69bd7adc0e93e2314723e732 INFO: 172.21.0.1:51706 - "POST /node/69bd7adc0e93e2314723e732 HTTP/1.0" 202 Accepted 2026-03-21 04:53:18,795 - lava_callback - INFO - Starting async job submit for node 69bd7adc0e93e2314723e732 2026-03-21 04:53:18,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:53:18,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd7adc0e93e2314723e732 HTTP/1.1" 200 None 2026-03-21 04:53:18,856 - lava_callback - DEBUG - Processing node 69bd7adc0e93e2314723e732: baseline-arm64-broonie 2026-03-21 04:53:19,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 04:53:19,332 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 04:53:19,333 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69bd7adc0e93e2314723e732/log.txt.gz 2026-03-21 04:53:19,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 04:53:19,753 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 04:53:19,755 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69bd7adc0e93e2314723e732/lava-logs.yaml 2026-03-21 04:53:19,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 04:53:19,941 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 04:53:19,943 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69bd7adc0e93e2314723e732/lava_callback.json.gz 2026-03-21 04:53:19,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:53:19,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd7adc0e93e2314723e732 HTTP/1.1" 200 None 2026-03-21 04:53:19,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:53:20,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-21 04:53:20,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:53:20,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd7adc0e93e2314723e732 HTTP/1.1" 200 None 2026-03-21 04:53:20,142 - lava_callback - INFO - Completed processing callback for node 69bd7adc0e93e2314723e732 INFO: 172.21.0.1:40008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53704 - "GET / HTTP/1.0" 200 OK 2026-03-21 04:53:37,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:53:37,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-21 04:53:41,396 - lava_callback - INFO - Received callback for node 69bb8146a1e2bfb1b3347965 INFO: 172.21.0.1:46246 - "POST /node/69bb8146a1e2bfb1b3347965 HTTP/1.0" 202 Accepted 2026-03-21 04:53:41,402 - lava_callback - INFO - Starting async job submit for node 69bb8146a1e2bfb1b3347965 2026-03-21 04:53:41,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:53:41,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb8146a1e2bfb1b3347965 HTTP/1.1" 200 None 2026-03-21 04:53:41,418 - lava_callback - DEBUG - Processing node 69bb8146a1e2bfb1b3347965: baseline-arm-clabbe 2026-03-21 04:53:41,419 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb8146a1e2bfb1b3347965 2026-03-21 04:53:41,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:53:41,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb8146a1e2bfb1b3347965 HTTP/1.1" 200 None 2026-03-21 04:53:41,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:53:41,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-21 04:53:41,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:53:41,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb8146a1e2bfb1b3347965 HTTP/1.1" 200 None 2026-03-21 04:53:41,482 - lava_callback - ERROR - Error processing callback for node 69bb8146a1e2bfb1b3347965: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45514 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45516 - "GET / HTTP/1.0" 200 OK 2026-03-21 04:54:21,340 - lava_callback - INFO - Received callback for node 69bb816aa1e2bfb1b3347974 INFO: 172.21.0.1:60866 - "POST /node/69bb816aa1e2bfb1b3347974 HTTP/1.0" 202 Accepted 2026-03-21 04:54:21,346 - lava_callback - INFO - Starting async job submit for node 69bb816aa1e2bfb1b3347974 2026-03-21 04:54:21,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:54:21,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb816aa1e2bfb1b3347974 HTTP/1.1" 200 None 2026-03-21 04:54:21,362 - lava_callback - DEBUG - Processing node 69bb816aa1e2bfb1b3347974: baseline-arm-clabbe 2026-03-21 04:54:21,362 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb816aa1e2bfb1b3347974 2026-03-21 04:54:21,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:54:21,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb816aa1e2bfb1b3347974 HTTP/1.1" 200 None 2026-03-21 04:54:21,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:54:21,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-21 04:54:21,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:54:21,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb816aa1e2bfb1b3347974 HTTP/1.1" 200 None 2026-03-21 04:54:21,424 - lava_callback - ERROR - Error processing callback for node 69bb816aa1e2bfb1b3347974: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58050 - "GET / HTTP/1.0" 200 OK 2026-03-21 04:54:41,365 - lava_callback - INFO - Received callback for node 69bb818ea1e2bfb1b334797d INFO: 172.21.0.1:44600 - "POST /node/69bb818ea1e2bfb1b334797d HTTP/1.0" 202 Accepted 2026-03-21 04:54:41,370 - lava_callback - INFO - Starting async job submit for node 69bb818ea1e2bfb1b334797d 2026-03-21 04:54:41,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:54:41,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb818ea1e2bfb1b334797d HTTP/1.1" 200 None 2026-03-21 04:54:41,387 - lava_callback - DEBUG - Processing node 69bb818ea1e2bfb1b334797d: baseline-arm-clabbe 2026-03-21 04:54:41,387 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb818ea1e2bfb1b334797d 2026-03-21 04:54:41,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:54:41,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb818ea1e2bfb1b334797d HTTP/1.1" 200 None 2026-03-21 04:54:41,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:54:41,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-21 04:54:41,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:54:41,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb818ea1e2bfb1b334797d HTTP/1.1" 200 None 2026-03-21 04:54:41,446 - lava_callback - ERROR - Error processing callback for node 69bb818ea1e2bfb1b334797d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56996 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45482 - "GET / HTTP/1.0" 200 OK 2026-03-21 04:55:30,960 - lava_callback - INFO - Received callback for node 69bb81ada1e2bfb1b334798a INFO: 172.21.0.1:44722 - "POST /node/69bb81ada1e2bfb1b334798a HTTP/1.0" 202 Accepted 2026-03-21 04:55:30,967 - lava_callback - INFO - Starting async job submit for node 69bb81ada1e2bfb1b334798a 2026-03-21 04:55:30,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:55:30,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb81ada1e2bfb1b334798a HTTP/1.1" 200 None 2026-03-21 04:55:30,992 - lava_callback - DEBUG - Processing node 69bb81ada1e2bfb1b334798a: baseline-arm-baylibre 2026-03-21 04:55:30,992 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb81ada1e2bfb1b334798a 2026-03-21 04:55:30,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:55:31,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb81ada1e2bfb1b334798a HTTP/1.1" 200 None 2026-03-21 04:55:31,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:55:31,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-21 04:55:31,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:55:31,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb81ada1e2bfb1b334798a HTTP/1.1" 200 None 2026-03-21 04:55:31,052 - lava_callback - ERROR - Error processing callback for node 69bb81ada1e2bfb1b334798a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50848 - "GET / HTTP/1.0" 200 OK 2026-03-21 04:55:50,519 - lava_callback - INFO - Received callback for node 69bb81d0a1e2bfb1b3347995 INFO: 172.21.0.1:58738 - "POST /node/69bb81d0a1e2bfb1b3347995 HTTP/1.0" 202 Accepted 2026-03-21 04:55:50,525 - lava_callback - INFO - Starting async job submit for node 69bb81d0a1e2bfb1b3347995 2026-03-21 04:55:50,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:55:50,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb81d0a1e2bfb1b3347995 HTTP/1.1" 200 None 2026-03-21 04:55:50,547 - lava_callback - DEBUG - Processing node 69bb81d0a1e2bfb1b3347995: baseline-arm-baylibre 2026-03-21 04:55:50,547 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb81d0a1e2bfb1b3347995 2026-03-21 04:55:50,548 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:55:50,563 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb81d0a1e2bfb1b3347995 HTTP/1.1" 200 None 2026-03-21 04:55:50,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:55:50,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-21 04:55:50,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:55:50,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb81d0a1e2bfb1b3347995 HTTP/1.1" 200 None 2026-03-21 04:55:50,608 - lava_callback - ERROR - Error processing callback for node 69bb81d0a1e2bfb1b3347995: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42292 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42298 - "GET / HTTP/1.0" 200 OK 2026-03-21 04:56:18,220 - lava_callback - INFO - Received callback for node 69bdd5c40e93e23147242ce7 INFO: 172.21.0.1:42308 - "POST /node/69bdd5c40e93e23147242ce7 HTTP/1.0" 202 Accepted 2026-03-21 04:56:18,226 - lava_callback - INFO - Starting async job submit for node 69bdd5c40e93e23147242ce7 2026-03-21 04:56:18,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:56:18,258 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd5c40e93e23147242ce7 HTTP/1.1" 200 None 2026-03-21 04:56:18,259 - lava_callback - DEBUG - Processing node 69bdd5c40e93e23147242ce7: baseline-arm64-broonie 2026-03-21 04:56:18,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 04:56:18,708 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 04:56:18,710 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69bdd5c40e93e23147242ce7/log.txt.gz 2026-03-21 04:56:18,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 04:56:19,177 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 04:56:19,179 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69bdd5c40e93e23147242ce7/lava-logs.yaml 2026-03-21 04:56:19,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 04:56:19,409 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 04:56:19,411 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69bdd5c40e93e23147242ce7/lava_callback.json.gz 2026-03-21 04:56:19,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:56:19,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd5c40e93e23147242ce7 HTTP/1.1" 200 None 2026-03-21 04:56:19,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:56:19,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-21 04:56:19,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:56:19,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdd5c40e93e23147242ce7 HTTP/1.1" 200 None 2026-03-21 04:56:19,558 - lava_callback - INFO - Completed processing callback for node 69bdd5c40e93e23147242ce7 INFO: 172.21.0.1:37998 - "GET / HTTP/1.0" 200 OK 2026-03-21 04:56:30,595 - lava_callback - INFO - Received callback for node 69bb81f4a1e2bfb1b33479a1 INFO: 172.21.0.1:35978 - "POST /node/69bb81f4a1e2bfb1b33479a1 HTTP/1.0" 202 Accepted 2026-03-21 04:56:30,600 - lava_callback - INFO - Starting async job submit for node 69bb81f4a1e2bfb1b33479a1 2026-03-21 04:56:30,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:56:30,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb81f4a1e2bfb1b33479a1 HTTP/1.1" 200 None 2026-03-21 04:56:30,616 - lava_callback - DEBUG - Processing node 69bb81f4a1e2bfb1b33479a1: baseline-arm-baylibre 2026-03-21 04:56:30,616 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb81f4a1e2bfb1b33479a1 2026-03-21 04:56:30,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:56:30,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb81f4a1e2bfb1b33479a1 HTTP/1.1" 200 None 2026-03-21 04:56:30,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:56:30,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-21 04:56:30,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:56:30,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb81f4a1e2bfb1b33479a1 HTTP/1.1" 200 None 2026-03-21 04:56:30,678 - lava_callback - ERROR - Error processing callback for node 69bb81f4a1e2bfb1b33479a1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35994 - "GET / HTTP/1.0" 200 OK 2026-03-21 04:56:37,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:56:37,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:33942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40008 - "GET / HTTP/1.0" 200 OK 2026-03-21 04:57:10,592 - lava_callback - INFO - Received callback for node 69bb8217a1e2bfb1b33479aa INFO: 172.21.0.1:41792 - "POST /node/69bb8217a1e2bfb1b33479aa HTTP/1.0" 202 Accepted 2026-03-21 04:57:10,597 - lava_callback - INFO - Starting async job submit for node 69bb8217a1e2bfb1b33479aa 2026-03-21 04:57:10,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:57:10,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb8217a1e2bfb1b33479aa HTTP/1.1" 200 None 2026-03-21 04:57:10,612 - lava_callback - DEBUG - Processing node 69bb8217a1e2bfb1b33479aa: baseline-arm-baylibre 2026-03-21 04:57:10,612 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb8217a1e2bfb1b33479aa 2026-03-21 04:57:10,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:57:10,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb8217a1e2bfb1b33479aa HTTP/1.1" 200 None 2026-03-21 04:57:10,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:57:10,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-21 04:57:10,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:57:10,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb8217a1e2bfb1b33479aa HTTP/1.1" 200 None 2026-03-21 04:57:10,671 - lava_callback - ERROR - Error processing callback for node 69bb8217a1e2bfb1b33479aa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41806 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57484 - "GET / HTTP/1.0" 200 OK 2026-03-21 04:57:50,663 - lava_callback - INFO - Received callback for node 69bb823ca1e2bfb1b33479b9 INFO: 172.21.0.1:44422 - "POST /node/69bb823ca1e2bfb1b33479b9 HTTP/1.0" 202 Accepted 2026-03-21 04:57:50,670 - lava_callback - INFO - Starting async job submit for node 69bb823ca1e2bfb1b33479b9 2026-03-21 04:57:50,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:57:50,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb823ca1e2bfb1b33479b9 HTTP/1.1" 200 None 2026-03-21 04:57:50,696 - lava_callback - DEBUG - Processing node 69bb823ca1e2bfb1b33479b9: baseline-arm-baylibre 2026-03-21 04:57:50,696 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb823ca1e2bfb1b33479b9 2026-03-21 04:57:50,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:57:50,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb823ca1e2bfb1b33479b9 HTTP/1.1" 200 None 2026-03-21 04:57:50,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:57:50,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-21 04:57:50,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:57:50,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb823ca1e2bfb1b33479b9 HTTP/1.1" 200 None 2026-03-21 04:57:50,768 - lava_callback - ERROR - Error processing callback for node 69bb823ca1e2bfb1b33479b9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43258 - "GET / HTTP/1.0" 200 OK 2026-03-21 04:58:10,686 - lava_callback - INFO - Received callback for node 69bb8261a1e2bfb1b33479c4 INFO: 172.21.0.1:56862 - "POST /node/69bb8261a1e2bfb1b33479c4 HTTP/1.0" 202 Accepted 2026-03-21 04:58:10,694 - lava_callback - INFO - Starting async job submit for node 69bb8261a1e2bfb1b33479c4 2026-03-21 04:58:10,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:58:10,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb8261a1e2bfb1b33479c4 HTTP/1.1" 200 None 2026-03-21 04:58:10,711 - lava_callback - DEBUG - Processing node 69bb8261a1e2bfb1b33479c4: baseline-arm-baylibre 2026-03-21 04:58:10,711 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb8261a1e2bfb1b33479c4 2026-03-21 04:58:10,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:58:10,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb8261a1e2bfb1b33479c4 HTTP/1.1" 200 None 2026-03-21 04:58:10,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:58:10,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-21 04:58:10,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:58:10,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb8261a1e2bfb1b33479c4 HTTP/1.1" 200 None 2026-03-21 04:58:10,768 - lava_callback - ERROR - Error processing callback for node 69bb8261a1e2bfb1b33479c4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56864 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45204 - "GET / HTTP/1.0" 200 OK 2026-03-21 04:58:50,722 - lava_callback - INFO - Received callback for node 69bb8282a1e2bfb1b33479cf INFO: 172.21.0.1:36478 - "POST /node/69bb8282a1e2bfb1b33479cf HTTP/1.0" 202 Accepted 2026-03-21 04:58:50,727 - lava_callback - INFO - Starting async job submit for node 69bb8282a1e2bfb1b33479cf 2026-03-21 04:58:50,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:58:50,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb8282a1e2bfb1b33479cf HTTP/1.1" 200 None 2026-03-21 04:58:50,744 - lava_callback - DEBUG - Processing node 69bb8282a1e2bfb1b33479cf: baseline-arm-baylibre 2026-03-21 04:58:50,744 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb8282a1e2bfb1b33479cf 2026-03-21 04:58:50,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:58:50,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb8282a1e2bfb1b33479cf HTTP/1.1" 200 None 2026-03-21 04:58:50,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:58:50,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-21 04:58:50,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:58:50,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb8282a1e2bfb1b33479cf HTTP/1.1" 200 None 2026-03-21 04:58:50,805 - lava_callback - ERROR - Error processing callback for node 69bb8282a1e2bfb1b33479cf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50326 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50328 - "GET / HTTP/1.0" 200 OK 2026-03-21 04:59:18,539 - lava_callback - INFO - Received callback for node 69bdf4bdd54b5122738c6a52 INFO: 172.21.0.1:50340 - "POST /node/69bdf4bdd54b5122738c6a52 HTTP/1.0" 202 Accepted 2026-03-21 04:59:18,545 - lava_callback - INFO - Starting async job submit for node 69bdf4bdd54b5122738c6a52 2026-03-21 04:59:18,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:59:18,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf4bdd54b5122738c6a52 HTTP/1.1" 200 None 2026-03-21 04:59:18,584 - lava_callback - DEBUG - Processing node 69bdf4bdd54b5122738c6a52: baseline-arm64-broonie 2026-03-21 04:59:18,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 04:59:19,045 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 04:59:19,047 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69bdf4bdd54b5122738c6a52/log.txt.gz 2026-03-21 04:59:19,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 04:59:19,550 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 04:59:19,552 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69bdf4bdd54b5122738c6a52/lava-logs.yaml 2026-03-21 04:59:19,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 04:59:19,697 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 04:59:19,698 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69bdf4bdd54b5122738c6a52/lava_callback.json.gz 2026-03-21 04:59:19,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:59:19,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf4bdd54b5122738c6a52 HTTP/1.1" 200 None 2026-03-21 04:59:19,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:59:19,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-21 04:59:19,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:59:19,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdf4bdd54b5122738c6a52 HTTP/1.1" 200 None 2026-03-21 04:59:19,837 - lava_callback - INFO - Completed processing callback for node 69bdf4bdd54b5122738c6a52 INFO: 172.21.0.1:39854 - "GET / HTTP/1.0" 200 OK 2026-03-21 04:59:30,771 - lava_callback - INFO - Received callback for node 69bb82a8a1e2bfb1b33479dd INFO: 172.21.0.1:44342 - "POST /node/69bb82a8a1e2bfb1b33479dd HTTP/1.0" 202 Accepted 2026-03-21 04:59:30,776 - lava_callback - INFO - Starting async job submit for node 69bb82a8a1e2bfb1b33479dd 2026-03-21 04:59:30,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:59:30,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb82a8a1e2bfb1b33479dd HTTP/1.1" 200 None 2026-03-21 04:59:30,792 - lava_callback - DEBUG - Processing node 69bb82a8a1e2bfb1b33479dd: baseline-arm-baylibre 2026-03-21 04:59:30,793 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb82a8a1e2bfb1b33479dd 2026-03-21 04:59:30,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:59:30,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb82a8a1e2bfb1b33479dd HTTP/1.1" 200 None 2026-03-21 04:59:30,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:59:30,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-21 04:59:30,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:59:30,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb82a8a1e2bfb1b33479dd HTTP/1.1" 200 None 2026-03-21 04:59:30,848 - lava_callback - ERROR - Error processing callback for node 69bb82a8a1e2bfb1b33479dd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44344 - "GET / HTTP/1.0" 200 OK 2026-03-21 04:59:37,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 04:59:37,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:58076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40708 - "GET / HTTP/1.0" 200 OK 2026-03-21 05:00:10,769 - lava_callback - INFO - Received callback for node 69bb82cba1e2bfb1b33479e6 INFO: 172.21.0.1:54806 - "POST /node/69bb82cba1e2bfb1b33479e6 HTTP/1.0" 202 Accepted 2026-03-21 05:00:10,775 - lava_callback - INFO - Starting async job submit for node 69bb82cba1e2bfb1b33479e6 2026-03-21 05:00:10,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:00:10,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb82cba1e2bfb1b33479e6 HTTP/1.1" 200 None 2026-03-21 05:00:10,790 - lava_callback - DEBUG - Processing node 69bb82cba1e2bfb1b33479e6: baseline-arm-baylibre 2026-03-21 05:00:10,791 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb82cba1e2bfb1b33479e6 2026-03-21 05:00:10,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:00:10,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb82cba1e2bfb1b33479e6 HTTP/1.1" 200 None 2026-03-21 05:00:10,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:00:10,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-21 05:00:10,830 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:00:10,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb82cba1e2bfb1b33479e6 HTTP/1.1" 200 None 2026-03-21 05:00:10,857 - lava_callback - ERROR - Error processing callback for node 69bb82cba1e2bfb1b33479e6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54810 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42482 - "GET / HTTP/1.0" 200 OK 2026-03-21 05:00:51,204 - lava_callback - INFO - Received callback for node 69bb82efa1e2bfb1b33479f2 INFO: 172.21.0.1:52060 - "POST /node/69bb82efa1e2bfb1b33479f2 HTTP/1.0" 202 Accepted 2026-03-21 05:00:51,210 - lava_callback - INFO - Starting async job submit for node 69bb82efa1e2bfb1b33479f2 2026-03-21 05:00:51,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:00:51,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb82efa1e2bfb1b33479f2 HTTP/1.1" 200 None 2026-03-21 05:00:51,227 - lava_callback - DEBUG - Processing node 69bb82efa1e2bfb1b33479f2: baseline-arm-baylibre 2026-03-21 05:00:51,227 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb82efa1e2bfb1b33479f2 2026-03-21 05:00:51,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:00:51,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb82efa1e2bfb1b33479f2 HTTP/1.1" 200 None 2026-03-21 05:00:51,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:00:51,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-21 05:00:51,270 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:00:51,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb82efa1e2bfb1b33479f2 HTTP/1.1" 200 None 2026-03-21 05:00:51,293 - lava_callback - ERROR - Error processing callback for node 69bb82efa1e2bfb1b33479f2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35794 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35798 - "GET / HTTP/1.0" 200 OK 2026-03-21 05:01:21,665 - lava_callback - INFO - Received callback for node 69bb8315a1e2bfb1b3347a01 INFO: 172.21.0.1:49174 - "POST /node/69bb8315a1e2bfb1b3347a01 HTTP/1.0" 202 Accepted 2026-03-21 05:01:21,671 - lava_callback - INFO - Starting async job submit for node 69bb8315a1e2bfb1b3347a01 2026-03-21 05:01:21,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:01:21,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb8315a1e2bfb1b3347a01 HTTP/1.1" 200 None 2026-03-21 05:01:21,688 - lava_callback - DEBUG - Processing node 69bb8315a1e2bfb1b3347a01: baseline-arm-clabbe 2026-03-21 05:01:21,689 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb8315a1e2bfb1b3347a01 2026-03-21 05:01:21,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:01:21,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb8315a1e2bfb1b3347a01 HTTP/1.1" 200 None 2026-03-21 05:01:21,704 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:01:21,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-21 05:01:21,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:01:21,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb8315a1e2bfb1b3347a01 HTTP/1.1" 200 None 2026-03-21 05:01:21,744 - lava_callback - ERROR - Error processing callback for node 69bb8315a1e2bfb1b3347a01: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54622 - "GET / HTTP/1.0" 200 OK 2026-03-21 05:02:01,631 - lava_callback - INFO - Received callback for node 69bb833aa1e2bfb1b3347a0d INFO: 172.21.0.1:42926 - "POST /node/69bb833aa1e2bfb1b3347a0d HTTP/1.0" 202 Accepted 2026-03-21 05:02:01,637 - lava_callback - INFO - Starting async job submit for node 69bb833aa1e2bfb1b3347a0d 2026-03-21 05:02:01,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:02:01,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb833aa1e2bfb1b3347a0d HTTP/1.1" 200 None 2026-03-21 05:02:01,659 - lava_callback - DEBUG - Processing node 69bb833aa1e2bfb1b3347a0d: baseline-arm-clabbe 2026-03-21 05:02:01,659 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb833aa1e2bfb1b3347a0d 2026-03-21 05:02:01,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:02:01,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb833aa1e2bfb1b3347a0d HTTP/1.1" 200 None 2026-03-21 05:02:01,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:02:01,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-21 05:02:01,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:02:01,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb833aa1e2bfb1b3347a0d HTTP/1.1" 200 None 2026-03-21 05:02:01,717 - lava_callback - ERROR - Error processing callback for node 69bb833aa1e2bfb1b3347a0d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33740 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33752 - "GET / HTTP/1.0" 200 OK 2026-03-21 05:02:20,171 - lava_callback - INFO - Received callback for node 69be24c2d54b5122738c7143 INFO: 172.21.0.1:57664 - "POST /node/69be24c2d54b5122738c7143 HTTP/1.0" 202 Accepted 2026-03-21 05:02:20,178 - lava_callback - INFO - Starting async job submit for node 69be24c2d54b5122738c7143 2026-03-21 05:02:20,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:02:20,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be24c2d54b5122738c7143 HTTP/1.1" 200 None 2026-03-21 05:02:20,212 - lava_callback - DEBUG - Processing node 69be24c2d54b5122738c7143: baseline-arm64-broonie 2026-03-21 05:02:20,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 05:02:20,708 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 05:02:20,710 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69be24c2d54b5122738c7143/log.txt.gz 2026-03-21 05:02:20,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 05:02:21,174 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 05:02:21,176 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69be24c2d54b5122738c7143/lava-logs.yaml 2026-03-21 05:02:21,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 05:02:21,349 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 05:02:21,351 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69be24c2d54b5122738c7143/lava_callback.json.gz 2026-03-21 05:02:21,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:02:21,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be24c2d54b5122738c7143 HTTP/1.1" 200 None 2026-03-21 05:02:21,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:02:21,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-21 05:02:21,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:02:21,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be24c2d54b5122738c7143 HTTP/1.1" 200 None 2026-03-21 05:02:21,495 - lava_callback - INFO - Completed processing callback for node 69be24c2d54b5122738c7143 INFO: 172.21.0.1:57676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49782 - "GET / HTTP/1.0" 200 OK 2026-03-21 05:02:37,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:02:37,158 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-21 05:02:41,670 - lava_callback - INFO - Received callback for node 69bb835fa1e2bfb1b3347a18 INFO: 172.21.0.1:52846 - "POST /node/69bb835fa1e2bfb1b3347a18 HTTP/1.0" 202 Accepted 2026-03-21 05:02:41,675 - lava_callback - INFO - Starting async job submit for node 69bb835fa1e2bfb1b3347a18 2026-03-21 05:02:41,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:02:41,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb835fa1e2bfb1b3347a18 HTTP/1.1" 200 None 2026-03-21 05:02:41,699 - lava_callback - DEBUG - Processing node 69bb835fa1e2bfb1b3347a18: baseline-arm-clabbe 2026-03-21 05:02:41,700 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb835fa1e2bfb1b3347a18 2026-03-21 05:02:41,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:02:41,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb835fa1e2bfb1b3347a18 HTTP/1.1" 200 None 2026-03-21 05:02:41,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:02:41,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-21 05:02:41,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:02:41,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb835fa1e2bfb1b3347a18 HTTP/1.1" 200 None 2026-03-21 05:02:41,783 - lava_callback - ERROR - Error processing callback for node 69bb835fa1e2bfb1b3347a18: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48544 - "GET / HTTP/1.0" 200 OK 2026-03-21 05:03:01,685 - lava_callback - INFO - Received callback for node 69bb8383a1e2bfb1b3347a24 INFO: 172.21.0.1:57674 - "POST /node/69bb8383a1e2bfb1b3347a24 HTTP/1.0" 202 Accepted 2026-03-21 05:03:01,691 - lava_callback - INFO - Starting async job submit for node 69bb8383a1e2bfb1b3347a24 2026-03-21 05:03:01,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:03:01,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb8383a1e2bfb1b3347a24 HTTP/1.1" 200 None 2026-03-21 05:03:01,706 - lava_callback - DEBUG - Processing node 69bb8383a1e2bfb1b3347a24: baseline-arm-clabbe 2026-03-21 05:03:01,707 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb8383a1e2bfb1b3347a24 2026-03-21 05:03:01,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:03:01,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb8383a1e2bfb1b3347a24 HTTP/1.1" 200 None 2026-03-21 05:03:01,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:03:01,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-21 05:03:01,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:03:01,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb8383a1e2bfb1b3347a24 HTTP/1.1" 200 None 2026-03-21 05:03:01,762 - lava_callback - ERROR - Error processing callback for node 69bb8383a1e2bfb1b3347a24: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59340 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34084 - "GET / HTTP/1.0" 200 OK 2026-03-21 05:03:41,779 - lava_callback - INFO - Received callback for node 69bb83a7a1e2bfb1b3347a2e INFO: 172.21.0.1:41086 - "POST /node/69bb83a7a1e2bfb1b3347a2e HTTP/1.0" 202 Accepted 2026-03-21 05:03:41,784 - lava_callback - INFO - Starting async job submit for node 69bb83a7a1e2bfb1b3347a2e 2026-03-21 05:03:41,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:03:41,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb83a7a1e2bfb1b3347a2e HTTP/1.1" 200 None 2026-03-21 05:03:41,809 - lava_callback - DEBUG - Processing node 69bb83a7a1e2bfb1b3347a2e: baseline-arm-clabbe 2026-03-21 05:03:41,809 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb83a7a1e2bfb1b3347a2e 2026-03-21 05:03:41,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:03:41,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb83a7a1e2bfb1b3347a2e HTTP/1.1" 200 None 2026-03-21 05:03:41,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:03:41,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-21 05:03:41,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:03:41,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb83a7a1e2bfb1b3347a2e HTTP/1.1" 200 None 2026-03-21 05:03:41,864 - lava_callback - ERROR - Error processing callback for node 69bb83a7a1e2bfb1b3347a2e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33842 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33846 - "GET / HTTP/1.0" 200 OK 2026-03-21 05:04:21,723 - lava_callback - INFO - Received callback for node 69bb83cba1e2bfb1b3347a47 INFO: 172.21.0.1:36602 - "POST /node/69bb83cba1e2bfb1b3347a47 HTTP/1.0" 202 Accepted 2026-03-21 05:04:21,729 - lava_callback - INFO - Starting async job submit for node 69bb83cba1e2bfb1b3347a47 2026-03-21 05:04:21,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:04:21,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb83cba1e2bfb1b3347a47 HTTP/1.1" 200 None 2026-03-21 05:04:21,746 - lava_callback - DEBUG - Processing node 69bb83cba1e2bfb1b3347a47: baseline-arm-clabbe 2026-03-21 05:04:21,747 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb83cba1e2bfb1b3347a47 2026-03-21 05:04:21,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:04:21,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb83cba1e2bfb1b3347a47 HTTP/1.1" 200 None 2026-03-21 05:04:21,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:04:21,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-21 05:04:21,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:04:21,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb83cba1e2bfb1b3347a47 HTTP/1.1" 200 None 2026-03-21 05:04:21,798 - lava_callback - ERROR - Error processing callback for node 69bb83cba1e2bfb1b3347a47: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53580 - "GET / HTTP/1.0" 200 OK 2026-03-21 05:05:01,709 - lava_callback - INFO - Received callback for node 69bb83f0a1e2bfb1b3347a50 INFO: 172.21.0.1:33634 - "POST /node/69bb83f0a1e2bfb1b3347a50 HTTP/1.0" 202 Accepted 2026-03-21 05:05:01,714 - lava_callback - INFO - Starting async job submit for node 69bb83f0a1e2bfb1b3347a50 2026-03-21 05:05:01,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:05:01,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb83f0a1e2bfb1b3347a50 HTTP/1.1" 200 None 2026-03-21 05:05:01,729 - lava_callback - DEBUG - Processing node 69bb83f0a1e2bfb1b3347a50: baseline-arm-clabbe 2026-03-21 05:05:01,730 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb83f0a1e2bfb1b3347a50 2026-03-21 05:05:01,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:05:01,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb83f0a1e2bfb1b3347a50 HTTP/1.1" 200 None 2026-03-21 05:05:01,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:05:01,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-21 05:05:01,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:05:01,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb83f0a1e2bfb1b3347a50 HTTP/1.1" 200 None 2026-03-21 05:05:01,781 - lava_callback - ERROR - Error processing callback for node 69bb83f0a1e2bfb1b3347a50: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54116 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54128 - "GET / HTTP/1.0" 200 OK 2026-03-21 05:05:18,617 - lava_callback - INFO - Received callback for node 69be25ead54b5122738c71ae INFO: 172.21.0.1:54136 - "POST /node/69be25ead54b5122738c71ae HTTP/1.0" 202 Accepted 2026-03-21 05:05:18,623 - lava_callback - INFO - Starting async job submit for node 69be25ead54b5122738c71ae 2026-03-21 05:05:18,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:05:18,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be25ead54b5122738c71ae HTTP/1.1" 200 None 2026-03-21 05:05:18,655 - lava_callback - DEBUG - Processing node 69be25ead54b5122738c71ae: baseline-arm64-broonie 2026-03-21 05:05:18,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 05:05:19,141 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 05:05:19,142 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69be25ead54b5122738c71ae/log.txt.gz 2026-03-21 05:05:19,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 05:05:19,564 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 05:05:19,566 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69be25ead54b5122738c71ae/lava-logs.yaml 2026-03-21 05:05:19,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 05:05:19,723 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 05:05:19,725 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69be25ead54b5122738c71ae/lava_callback.json.gz 2026-03-21 05:05:19,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:05:19,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be25ead54b5122738c71ae HTTP/1.1" 200 None 2026-03-21 05:05:19,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:05:19,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-21 05:05:19,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:05:19,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be25ead54b5122738c71ae HTTP/1.1" 200 None 2026-03-21 05:05:19,906 - lava_callback - INFO - Completed processing callback for node 69be25ead54b5122738c71ae 2026-03-21 05:05:20,308 - lava_callback - INFO - Received callback for node 69be265bd54b5122738c71c5 INFO: 172.21.0.1:55830 - "POST /node/69be265bd54b5122738c71c5 HTTP/1.0" 202 Accepted 2026-03-21 05:05:20,320 - lava_callback - INFO - Starting async job submit for node 69be265bd54b5122738c71c5 2026-03-21 05:05:20,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:05:20,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be265bd54b5122738c71c5 HTTP/1.1" 200 None 2026-03-21 05:05:20,424 - lava_callback - DEBUG - Processing node 69be265bd54b5122738c71c5: baseline-arm64-broonie 2026-03-21 05:05:20,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 05:05:20,976 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 05:05:20,977 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69be265bd54b5122738c71c5/log.txt.gz 2026-03-21 05:05:21,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 05:05:21,411 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 05:05:21,413 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69be265bd54b5122738c71c5/lava-logs.yaml 2026-03-21 05:05:21,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 05:05:21,546 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 05:05:21,548 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69be265bd54b5122738c71c5/lava_callback.json.gz 2026-03-21 05:05:21,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:05:21,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be265bd54b5122738c71c5 HTTP/1.1" 200 None 2026-03-21 05:05:21,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:05:21,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-21 05:05:21,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:05:21,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be265bd54b5122738c71c5 HTTP/1.1" 200 None 2026-03-21 05:05:21,720 - lava_callback - INFO - Completed processing callback for node 69be265bd54b5122738c71c5 INFO: 172.21.0.1:55844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49410 - "GET / HTTP/1.0" 200 OK 2026-03-21 05:05:37,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:05:37,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-21 05:05:41,797 - lava_callback - INFO - Received callback for node 69bb8414a1e2bfb1b3347a5d INFO: 172.21.0.1:51080 - "POST /node/69bb8414a1e2bfb1b3347a5d HTTP/1.0" 202 Accepted 2026-03-21 05:05:41,802 - lava_callback - INFO - Starting async job submit for node 69bb8414a1e2bfb1b3347a5d 2026-03-21 05:05:41,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:05:41,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb8414a1e2bfb1b3347a5d HTTP/1.1" 200 None 2026-03-21 05:05:41,820 - lava_callback - DEBUG - Processing node 69bb8414a1e2bfb1b3347a5d: baseline-arm-clabbe 2026-03-21 05:05:41,820 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb8414a1e2bfb1b3347a5d 2026-03-21 05:05:41,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:05:41,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb8414a1e2bfb1b3347a5d HTTP/1.1" 200 None 2026-03-21 05:05:41,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:05:41,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-21 05:05:41,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:05:41,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb8414a1e2bfb1b3347a5d HTTP/1.1" 200 None 2026-03-21 05:05:41,875 - lava_callback - ERROR - Error processing callback for node 69bb8414a1e2bfb1b3347a5d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52768 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52778 - "GET / HTTP/1.0" 200 OK 2026-03-21 05:06:21,840 - lava_callback - INFO - Received callback for node 69bb8439a1e2bfb1b3347a69 INFO: 172.21.0.1:48768 - "POST /node/69bb8439a1e2bfb1b3347a69 HTTP/1.0" 202 Accepted 2026-03-21 05:06:21,846 - lava_callback - INFO - Starting async job submit for node 69bb8439a1e2bfb1b3347a69 2026-03-21 05:06:21,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:06:21,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb8439a1e2bfb1b3347a69 HTTP/1.1" 200 None 2026-03-21 05:06:21,862 - lava_callback - DEBUG - Processing node 69bb8439a1e2bfb1b3347a69: baseline-arm-clabbe 2026-03-21 05:06:21,862 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb8439a1e2bfb1b3347a69 2026-03-21 05:06:21,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:06:21,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb8439a1e2bfb1b3347a69 HTTP/1.1" 200 None 2026-03-21 05:06:21,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:06:21,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-21 05:06:21,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:06:21,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb8439a1e2bfb1b3347a69 HTTP/1.1" 200 None 2026-03-21 05:06:21,918 - lava_callback - ERROR - Error processing callback for node 69bb8439a1e2bfb1b3347a69: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47866 - "GET / HTTP/1.0" 200 OK 2026-03-21 05:06:41,833 - lava_callback - INFO - Received callback for node 69bb845ea1e2bfb1b3347a74 INFO: 172.21.0.1:45464 - "POST /node/69bb845ea1e2bfb1b3347a74 HTTP/1.0" 202 Accepted 2026-03-21 05:06:41,838 - lava_callback - INFO - Starting async job submit for node 69bb845ea1e2bfb1b3347a74 2026-03-21 05:06:41,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:06:41,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb845ea1e2bfb1b3347a74 HTTP/1.1" 200 None 2026-03-21 05:06:41,854 - lava_callback - DEBUG - Processing node 69bb845ea1e2bfb1b3347a74: baseline-arm-clabbe 2026-03-21 05:06:41,854 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb845ea1e2bfb1b3347a74 2026-03-21 05:06:41,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:06:41,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb845ea1e2bfb1b3347a74 HTTP/1.1" 200 None 2026-03-21 05:06:41,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:06:41,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-21 05:06:41,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:06:41,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb845ea1e2bfb1b3347a74 HTTP/1.1" 200 None 2026-03-21 05:06:41,905 - lava_callback - ERROR - Error processing callback for node 69bb845ea1e2bfb1b3347a74: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40766 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49664 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54846 - "GET / HTTP/1.0" 200 OK 2026-03-21 05:08:31,380 - lava_callback - INFO - Received callback for node 69bb84c8a1e2bfb1b3347a8a INFO: 172.21.0.1:33518 - "POST /node/69bb84c8a1e2bfb1b3347a8a HTTP/1.0" 202 Accepted 2026-03-21 05:08:31,386 - lava_callback - INFO - Starting async job submit for node 69bb84c8a1e2bfb1b3347a8a 2026-03-21 05:08:31,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:08:31,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb84c8a1e2bfb1b3347a8a HTTP/1.1" 200 None 2026-03-21 05:08:31,415 - lava_callback - DEBUG - Processing node 69bb84c8a1e2bfb1b3347a8a: baseline-arm64-baylibre 2026-03-21 05:08:31,415 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb84c8a1e2bfb1b3347a8a 2026-03-21 05:08:31,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:08:31,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb84c8a1e2bfb1b3347a8a HTTP/1.1" 200 None 2026-03-21 05:08:31,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:08:31,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d510b8ad4124cec3566 HTTP/1.1" 200 None 2026-03-21 05:08:31,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:08:31,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb84c8a1e2bfb1b3347a8a HTTP/1.1" 200 None 2026-03-21 05:08:31,484 - lava_callback - ERROR - Error processing callback for node 69bb84c8a1e2bfb1b3347a8a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43566 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33978 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60438 - "GET / HTTP/1.0" 200 OK 2026-03-21 05:10:51,392 - lava_callback - INFO - Received callback for node 69bb8559a1e2bfb1b3347aa0 INFO: 172.21.0.1:56790 - "POST /node/69bb8559a1e2bfb1b3347aa0 HTTP/1.0" 202 Accepted 2026-03-21 05:10:51,397 - lava_callback - INFO - Starting async job submit for node 69bb8559a1e2bfb1b3347aa0 2026-03-21 05:10:51,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:10:51,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb8559a1e2bfb1b3347aa0 HTTP/1.1" 200 None 2026-03-21 05:10:51,413 - lava_callback - DEBUG - Processing node 69bb8559a1e2bfb1b3347aa0: baseline-arm64-baylibre 2026-03-21 05:10:51,413 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb8559a1e2bfb1b3347aa0 2026-03-21 05:10:51,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:10:51,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb8559a1e2bfb1b3347aa0 HTTP/1.1" 200 None 2026-03-21 05:10:51,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:10:51,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d510b8ad4124cec3566 HTTP/1.1" 200 None 2026-03-21 05:10:51,461 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:10:51,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb8559a1e2bfb1b3347aa0 HTTP/1.1" 200 None 2026-03-21 05:10:51,490 - lava_callback - ERROR - Error processing callback for node 69bb8559a1e2bfb1b3347aa0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56804 - "GET / HTTP/1.0" 200 OK 2026-03-21 05:10:58,785 - lava_callback - INFO - Received callback for node 69be27a6d54b5122738c723c INFO: 172.21.0.1:56810 - "POST /node/69be27a6d54b5122738c723c HTTP/1.0" 202 Accepted 2026-03-21 05:10:58,791 - lava_callback - INFO - Starting async job submit for node 69be27a6d54b5122738c723c 2026-03-21 05:10:58,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:10:58,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be27a6d54b5122738c723c HTTP/1.1" 200 None 2026-03-21 05:10:58,824 - lava_callback - DEBUG - Processing node 69be27a6d54b5122738c723c: baseline-arm64-broonie 2026-03-21 05:10:59,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 05:10:59,337 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 05:10:59,338 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69be27a6d54b5122738c723c/log.txt.gz 2026-03-21 05:10:59,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 05:10:59,738 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 05:10:59,740 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69be27a6d54b5122738c723c/lava-logs.yaml 2026-03-21 05:10:59,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 05:10:59,993 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 05:10:59,995 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69be27a6d54b5122738c723c/lava_callback.json.gz 2026-03-21 05:11:00,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:11:00,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be27a6d54b5122738c723c HTTP/1.1" 200 None 2026-03-21 05:11:00,054 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:11:00,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-21 05:11:00,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:11:00,093 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be27a6d54b5122738c723c HTTP/1.1" 200 None 2026-03-21 05:11:00,143 - lava_callback - INFO - Completed processing callback for node 69be27a6d54b5122738c723c INFO: 172.21.0.1:47906 - "GET / HTTP/1.0" 200 OK 2026-03-21 05:11:07,181 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:11:07,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:47922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37178 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43258 - "GET / HTTP/1.0" 200 OK 2026-03-21 05:12:11,483 - lava_callback - INFO - Received callback for node 69bb85a9a1e2bfb1b3347aae INFO: 172.21.0.1:45390 - "POST /node/69bb85a9a1e2bfb1b3347aae HTTP/1.0" 202 Accepted 2026-03-21 05:12:11,488 - lava_callback - INFO - Starting async job submit for node 69bb85a9a1e2bfb1b3347aae 2026-03-21 05:12:11,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:12:11,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb85a9a1e2bfb1b3347aae HTTP/1.1" 200 None 2026-03-21 05:12:11,505 - lava_callback - DEBUG - Processing node 69bb85a9a1e2bfb1b3347aae: baseline-arm64-baylibre 2026-03-21 05:12:11,505 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb85a9a1e2bfb1b3347aae 2026-03-21 05:12:11,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:12:11,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb85a9a1e2bfb1b3347aae HTTP/1.1" 200 None 2026-03-21 05:12:11,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:12:11,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d510b8ad4124cec3566 HTTP/1.1" 200 None 2026-03-21 05:12:11,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:12:11,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb85a9a1e2bfb1b3347aae HTTP/1.1" 200 None 2026-03-21 05:12:11,575 - lava_callback - ERROR - Error processing callback for node 69bb85a9a1e2bfb1b3347aae: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45404 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60130 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43346 - "GET / HTTP/1.0" 200 OK 2026-03-21 05:13:31,528 - lava_callback - INFO - Received callback for node 69bb85f8a1e2bfb1b3347abc INFO: 172.21.0.1:42642 - "POST /node/69bb85f8a1e2bfb1b3347abc HTTP/1.0" 202 Accepted 2026-03-21 05:13:31,534 - lava_callback - INFO - Starting async job submit for node 69bb85f8a1e2bfb1b3347abc 2026-03-21 05:13:31,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:13:31,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb85f8a1e2bfb1b3347abc HTTP/1.1" 200 None 2026-03-21 05:13:31,551 - lava_callback - DEBUG - Processing node 69bb85f8a1e2bfb1b3347abc: baseline-arm64-baylibre 2026-03-21 05:13:31,551 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb85f8a1e2bfb1b3347abc 2026-03-21 05:13:31,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:13:31,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb85f8a1e2bfb1b3347abc HTTP/1.1" 200 None 2026-03-21 05:13:31,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:13:31,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d510b8ad4124cec3566 HTTP/1.1" 200 None 2026-03-21 05:13:31,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:13:31,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb85f8a1e2bfb1b3347abc HTTP/1.1" 200 None 2026-03-21 05:13:31,606 - lava_callback - ERROR - Error processing callback for node 69bb85f8a1e2bfb1b3347abc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53646 - "GET / HTTP/1.0" 200 OK 2026-03-21 05:13:58,817 - lava_callback - INFO - Received callback for node 69be27f1d54b5122738c724a INFO: 172.21.0.1:53654 - "POST /node/69be27f1d54b5122738c724a HTTP/1.0" 202 Accepted 2026-03-21 05:13:58,823 - lava_callback - INFO - Starting async job submit for node 69be27f1d54b5122738c724a 2026-03-21 05:13:58,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:13:58,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be27f1d54b5122738c724a HTTP/1.1" 200 None 2026-03-21 05:13:58,855 - lava_callback - DEBUG - Processing node 69be27f1d54b5122738c724a: baseline-arm64-broonie 2026-03-21 05:13:59,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 05:13:59,433 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 05:13:59,435 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69be27f1d54b5122738c724a/log.txt.gz 2026-03-21 05:13:59,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 05:13:59,869 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 05:13:59,871 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69be27f1d54b5122738c724a/lava-logs.yaml 2026-03-21 05:13:59,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 05:14:00,031 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 05:14:00,033 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69be27f1d54b5122738c724a/lava_callback.json.gz 2026-03-21 05:14:00,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:14:00,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be27f1d54b5122738c724a HTTP/1.1" 200 None 2026-03-21 05:14:00,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:14:00,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-21 05:14:00,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:14:00,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be27f1d54b5122738c724a HTTP/1.1" 200 None 2026-03-21 05:14:00,170 - lava_callback - INFO - Completed processing callback for node 69be27f1d54b5122738c724a INFO: 172.21.0.1:36982 - "GET / HTTP/1.0" 200 OK 2026-03-21 05:14:07,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:14:07,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:36990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59368 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34748 - "GET / HTTP/1.0" 200 OK 2026-03-21 05:14:51,602 - lava_callback - INFO - Received callback for node 69bb8649a1e2bfb1b3347ac9 INFO: 172.21.0.1:33686 - "POST /node/69bb8649a1e2bfb1b3347ac9 HTTP/1.0" 202 Accepted 2026-03-21 05:14:51,607 - lava_callback - INFO - Starting async job submit for node 69bb8649a1e2bfb1b3347ac9 2026-03-21 05:14:51,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:14:51,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb8649a1e2bfb1b3347ac9 HTTP/1.1" 200 None 2026-03-21 05:14:51,623 - lava_callback - DEBUG - Processing node 69bb8649a1e2bfb1b3347ac9: baseline-arm64-baylibre 2026-03-21 05:14:51,624 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb8649a1e2bfb1b3347ac9 2026-03-21 05:14:51,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:14:51,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb8649a1e2bfb1b3347ac9 HTTP/1.1" 200 None 2026-03-21 05:14:51,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:14:51,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d510b8ad4124cec3566 HTTP/1.1" 200 None 2026-03-21 05:14:51,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:14:51,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb8649a1e2bfb1b3347ac9 HTTP/1.1" 200 None 2026-03-21 05:14:51,678 - lava_callback - ERROR - Error processing callback for node 69bb8649a1e2bfb1b3347ac9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40940 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40150 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36336 - "GET / HTTP/1.0" 200 OK 2026-03-21 05:16:31,905 - lava_callback - INFO - Received callback for node 69bb869aa1e2bfb1b3347ade INFO: 172.21.0.1:41950 - "POST /node/69bb869aa1e2bfb1b3347ade HTTP/1.0" 202 Accepted 2026-03-21 05:16:31,910 - lava_callback - INFO - Starting async job submit for node 69bb869aa1e2bfb1b3347ade 2026-03-21 05:16:31,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:16:31,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb869aa1e2bfb1b3347ade HTTP/1.1" 200 None 2026-03-21 05:16:31,926 - lava_callback - DEBUG - Processing node 69bb869aa1e2bfb1b3347ade: baseline-arm64-baylibre 2026-03-21 05:16:31,927 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb869aa1e2bfb1b3347ade 2026-03-21 05:16:31,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:16:31,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb869aa1e2bfb1b3347ade HTTP/1.1" 200 None 2026-03-21 05:16:31,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:16:31,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d510b8ad4124cec3566 HTTP/1.1" 200 None 2026-03-21 05:16:31,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:16:31,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb869aa1e2bfb1b3347ade HTTP/1.1" 200 None 2026-03-21 05:16:31,982 - lava_callback - ERROR - Error processing callback for node 69bb869aa1e2bfb1b3347ade: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41132 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45148 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52642 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34872 - "GET / HTTP/1.0" 200 OK 2026-03-21 05:19:39,265 - lava_callback - INFO - Received callback for node 69be29b9d54b5122738c729d INFO: 172.21.0.1:34880 - "POST /node/69be29b9d54b5122738c729d HTTP/1.0" 202 Accepted 2026-03-21 05:19:39,271 - lava_callback - INFO - Starting async job submit for node 69be29b9d54b5122738c729d 2026-03-21 05:19:39,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:19:39,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be29b9d54b5122738c729d HTTP/1.1" 200 None 2026-03-21 05:19:39,310 - lava_callback - DEBUG - Processing node 69be29b9d54b5122738c729d: baseline-arm64-broonie 2026-03-21 05:19:39,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 05:19:39,771 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 05:19:39,773 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69be29b9d54b5122738c729d/log.txt.gz 2026-03-21 05:19:39,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 05:19:40,194 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 05:19:40,196 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69be29b9d54b5122738c729d/lava-logs.yaml 2026-03-21 05:19:40,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 05:19:40,351 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 05:19:40,353 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69be29b9d54b5122738c729d/lava_callback.json.gz 2026-03-21 05:19:40,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:19:40,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be29b9d54b5122738c729d HTTP/1.1" 200 None 2026-03-21 05:19:40,405 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:19:40,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-21 05:19:40,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:19:40,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be29b9d54b5122738c729d HTTP/1.1" 200 None 2026-03-21 05:19:40,501 - lava_callback - INFO - Completed processing callback for node 69be29b9d54b5122738c729d INFO: 172.21.0.1:34014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49114 - "GET / HTTP/1.0" 200 OK 2026-03-21 05:20:07,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:20:07,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:49116 - "GET / HTTP/1.0" 200 OK 2026-03-21 05:20:12,007 - lava_callback - INFO - Received callback for node 69bb8789a1e2bfb1b3347aff INFO: 172.21.0.1:38262 - "POST /node/69bb8789a1e2bfb1b3347aff HTTP/1.0" 202 Accepted 2026-03-21 05:20:12,013 - lava_callback - INFO - Starting async job submit for node 69bb8789a1e2bfb1b3347aff 2026-03-21 05:20:12,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:20:12,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb8789a1e2bfb1b3347aff HTTP/1.1" 200 None 2026-03-21 05:20:12,037 - lava_callback - DEBUG - Processing node 69bb8789a1e2bfb1b3347aff: baseline-arm64-baylibre 2026-03-21 05:20:12,037 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb8789a1e2bfb1b3347aff 2026-03-21 05:20:12,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:20:12,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb8789a1e2bfb1b3347aff HTTP/1.1" 200 None 2026-03-21 05:20:12,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:20:12,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d510b8ad4124cec3566 HTTP/1.1" 200 None 2026-03-21 05:20:12,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:20:12,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb8789a1e2bfb1b3347aff HTTP/1.1" 200 None 2026-03-21 05:20:12,130 - lava_callback - ERROR - Error processing callback for node 69bb8789a1e2bfb1b3347aff: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38276 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57196 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52766 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35014 - "GET / HTTP/1.0" 200 OK 2026-03-21 05:22:52,114 - lava_callback - INFO - Received callback for node 69bb8828a1e2bfb1b3347b19 INFO: 172.21.0.1:33904 - "POST /node/69bb8828a1e2bfb1b3347b19 HTTP/1.0" 202 Accepted 2026-03-21 05:22:52,119 - lava_callback - INFO - Starting async job submit for node 69bb8828a1e2bfb1b3347b19 2026-03-21 05:22:52,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:22:52,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb8828a1e2bfb1b3347b19 HTTP/1.1" 200 None 2026-03-21 05:22:52,149 - lava_callback - DEBUG - Processing node 69bb8828a1e2bfb1b3347b19: baseline-arm64-baylibre 2026-03-21 05:22:52,149 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb8828a1e2bfb1b3347b19 2026-03-21 05:22:52,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:22:52,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb8828a1e2bfb1b3347b19 HTTP/1.1" 200 None 2026-03-21 05:22:52,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:22:52,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d510b8ad4124cec3566 HTTP/1.1" 200 None 2026-03-21 05:22:52,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:22:52,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb8828a1e2bfb1b3347b19 HTTP/1.1" 200 None 2026-03-21 05:22:52,223 - lava_callback - ERROR - Error processing callback for node 69bb8828a1e2bfb1b3347b19: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58262 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60208 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42618 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41782 - "GET / HTTP/1.0" 200 OK 2026-03-21 05:25:42,772 - lava_callback - INFO - Received callback for node 69bb88cfa1e2bfb1b3347b31 INFO: 172.21.0.1:35204 - "POST /node/69bb88cfa1e2bfb1b3347b31 HTTP/1.0" 202 Accepted 2026-03-21 05:25:42,778 - lava_callback - INFO - Starting async job submit for node 69bb88cfa1e2bfb1b3347b31 2026-03-21 05:25:42,779 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:25:42,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb88cfa1e2bfb1b3347b31 HTTP/1.1" 200 None 2026-03-21 05:25:42,794 - lava_callback - DEBUG - Processing node 69bb88cfa1e2bfb1b3347b31: baseline-arm64-clabbe 2026-03-21 05:25:42,794 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb88cfa1e2bfb1b3347b31 2026-03-21 05:25:42,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:25:42,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb88cfa1e2bfb1b3347b31 HTTP/1.1" 200 None 2026-03-21 05:25:42,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:25:42,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d510b8ad4124cec3566 HTTP/1.1" 200 None 2026-03-21 05:25:42,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:25:42,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb88cfa1e2bfb1b3347b31 HTTP/1.1" 200 None 2026-03-21 05:25:42,857 - lava_callback - ERROR - Error processing callback for node 69bb88cfa1e2bfb1b3347b31: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36440 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39442 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57886 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57888 - "GET / HTTP/1.0" 200 OK 2026-03-21 05:28:22,862 - lava_callback - INFO - Received callback for node 69bb896ca1e2bfb1b3347b50 INFO: 172.21.0.1:48726 - "POST /node/69bb896ca1e2bfb1b3347b50 HTTP/1.0" 202 Accepted 2026-03-21 05:28:22,868 - lava_callback - INFO - Starting async job submit for node 69bb896ca1e2bfb1b3347b50 2026-03-21 05:28:22,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:28:22,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb896ca1e2bfb1b3347b50 HTTP/1.1" 200 None 2026-03-21 05:28:22,884 - lava_callback - DEBUG - Processing node 69bb896ca1e2bfb1b3347b50: baseline-arm64-clabbe 2026-03-21 05:28:22,885 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb896ca1e2bfb1b3347b50 2026-03-21 05:28:22,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:28:22,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb896ca1e2bfb1b3347b50 HTTP/1.1" 200 None 2026-03-21 05:28:22,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:28:22,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d510b8ad4124cec3566 HTTP/1.1" 200 None 2026-03-21 05:28:22,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:28:22,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb896ca1e2bfb1b3347b50 HTTP/1.1" 200 None 2026-03-21 05:28:22,951 - lava_callback - ERROR - Error processing callback for node 69bb896ca1e2bfb1b3347b50: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35234 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44610 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50244 - "GET / HTTP/1.0" 200 OK 2026-03-21 05:31:03,099 - lava_callback - INFO - Received callback for node 69bb8a0ba1e2bfb1b3347b66 INFO: 172.21.0.1:48248 - "POST /node/69bb8a0ba1e2bfb1b3347b66 HTTP/1.0" 202 Accepted 2026-03-21 05:31:03,104 - lava_callback - INFO - Starting async job submit for node 69bb8a0ba1e2bfb1b3347b66 2026-03-21 05:31:03,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:31:03,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb8a0ba1e2bfb1b3347b66 HTTP/1.1" 200 None 2026-03-21 05:31:03,120 - lava_callback - DEBUG - Processing node 69bb8a0ba1e2bfb1b3347b66: baseline-arm64-clabbe 2026-03-21 05:31:03,120 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb8a0ba1e2bfb1b3347b66 2026-03-21 05:31:03,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:31:03,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb8a0ba1e2bfb1b3347b66 HTTP/1.1" 200 None 2026-03-21 05:31:03,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:31:03,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d510b8ad4124cec3566 HTTP/1.1" 200 None 2026-03-21 05:31:03,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:31:03,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb8a0ba1e2bfb1b3347b66 HTTP/1.1" 200 None 2026-03-21 05:31:03,187 - lava_callback - ERROR - Error processing callback for node 69bb8a0ba1e2bfb1b3347b66: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35752 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56870 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39112 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36610 - "GET / HTTP/1.0" 200 OK 2026-03-21 05:33:43,218 - lava_callback - INFO - Received callback for node 69bb8aaaa1e2bfb1b3347b79 INFO: 172.21.0.1:54156 - "POST /node/69bb8aaaa1e2bfb1b3347b79 HTTP/1.0" 202 Accepted 2026-03-21 05:33:43,224 - lava_callback - INFO - Starting async job submit for node 69bb8aaaa1e2bfb1b3347b79 2026-03-21 05:33:43,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:33:43,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb8aaaa1e2bfb1b3347b79 HTTP/1.1" 200 None 2026-03-21 05:33:43,254 - lava_callback - DEBUG - Processing node 69bb8aaaa1e2bfb1b3347b79: baseline-arm64-clabbe 2026-03-21 05:33:43,254 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb8aaaa1e2bfb1b3347b79 2026-03-21 05:33:43,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:33:43,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb8aaaa1e2bfb1b3347b79 HTTP/1.1" 200 None 2026-03-21 05:33:43,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:33:43,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d510b8ad4124cec3566 HTTP/1.1" 200 None 2026-03-21 05:33:43,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:33:43,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb8aaaa1e2bfb1b3347b79 HTTP/1.1" 200 None 2026-03-21 05:33:43,313 - lava_callback - ERROR - Error processing callback for node 69bb8aaaa1e2bfb1b3347b79: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60742 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36652 - "GET / HTTP/1.0" 200 OK 2026-03-21 05:34:52,746 - lava_callback - INFO - Received callback for node 69bb8af1a1e2bfb1b3347b86 INFO: 172.21.0.1:52200 - "POST /node/69bb8af1a1e2bfb1b3347b86 HTTP/1.0" 202 Accepted 2026-03-21 05:34:52,751 - lava_callback - INFO - Starting async job submit for node 69bb8af1a1e2bfb1b3347b86 2026-03-21 05:34:52,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:34:52,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb8af1a1e2bfb1b3347b86 HTTP/1.1" 200 None 2026-03-21 05:34:52,799 - lava_callback - DEBUG - Processing node 69bb8af1a1e2bfb1b3347b86: baseline-arm64-baylibre 2026-03-21 05:34:52,799 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb8af1a1e2bfb1b3347b86 2026-03-21 05:34:52,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:34:52,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb8af1a1e2bfb1b3347b86 HTTP/1.1" 200 None 2026-03-21 05:34:52,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:34:52,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d510b8ad4124cec3566 HTTP/1.1" 200 None 2026-03-21 05:34:52,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:34:52,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb8af1a1e2bfb1b3347b86 HTTP/1.1" 200 None 2026-03-21 05:34:52,859 - lava_callback - ERROR - Error processing callback for node 69bb8af1a1e2bfb1b3347b86: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49574 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39844 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39852 - "GET / HTTP/1.0" 200 OK 2026-03-21 05:36:23,218 - lava_callback - INFO - Received callback for node 69bb8b4aa1e2bfb1b3347b8f INFO: 172.21.0.1:60448 - "POST /node/69bb8b4aa1e2bfb1b3347b8f HTTP/1.0" 202 Accepted 2026-03-21 05:36:23,225 - lava_callback - INFO - Starting async job submit for node 69bb8b4aa1e2bfb1b3347b8f 2026-03-21 05:36:23,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:36:23,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb8b4aa1e2bfb1b3347b8f HTTP/1.1" 200 None 2026-03-21 05:36:23,242 - lava_callback - DEBUG - Processing node 69bb8b4aa1e2bfb1b3347b8f: baseline-arm64-clabbe 2026-03-21 05:36:23,242 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb8b4aa1e2bfb1b3347b8f 2026-03-21 05:36:23,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:36:23,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb8b4aa1e2bfb1b3347b8f HTTP/1.1" 200 None 2026-03-21 05:36:23,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:36:23,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d510b8ad4124cec3566 HTTP/1.1" 200 None 2026-03-21 05:36:23,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:36:23,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb8b4aa1e2bfb1b3347b8f HTTP/1.1" 200 None 2026-03-21 05:36:23,312 - lava_callback - ERROR - Error processing callback for node 69bb8b4aa1e2bfb1b3347b8f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59422 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50342 - "GET / HTTP/1.0" 200 OK 2026-03-21 05:37:32,813 - lava_callback - INFO - Received callback for node 69bb8b91a1e2bfb1b3347b9b INFO: 172.21.0.1:52280 - "POST /node/69bb8b91a1e2bfb1b3347b9b HTTP/1.0" 202 Accepted 2026-03-21 05:37:32,819 - lava_callback - INFO - Starting async job submit for node 69bb8b91a1e2bfb1b3347b9b 2026-03-21 05:37:32,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:37:32,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb8b91a1e2bfb1b3347b9b HTTP/1.1" 200 None 2026-03-21 05:37:32,836 - lava_callback - DEBUG - Processing node 69bb8b91a1e2bfb1b3347b9b: baseline-arm64-baylibre 2026-03-21 05:37:32,836 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb8b91a1e2bfb1b3347b9b 2026-03-21 05:37:32,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:37:32,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb8b91a1e2bfb1b3347b9b HTTP/1.1" 200 None 2026-03-21 05:37:32,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:37:32,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-21 05:37:32,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:37:32,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb8b91a1e2bfb1b3347b9b HTTP/1.1" 200 None 2026-03-21 05:37:32,899 - lava_callback - ERROR - Error processing callback for node 69bb8b91a1e2bfb1b3347b9b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57746 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33842 - "GET / HTTP/1.0" 200 OK 2026-03-21 05:39:03,399 - lava_callback - INFO - Received callback for node 69bb8be9a1e2bfb1b3347ba9 INFO: 172.21.0.1:51816 - "POST /node/69bb8be9a1e2bfb1b3347ba9 HTTP/1.0" 202 Accepted 2026-03-21 05:39:03,405 - lava_callback - INFO - Starting async job submit for node 69bb8be9a1e2bfb1b3347ba9 2026-03-21 05:39:03,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:39:03,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb8be9a1e2bfb1b3347ba9 HTTP/1.1" 200 None 2026-03-21 05:39:03,421 - lava_callback - DEBUG - Processing node 69bb8be9a1e2bfb1b3347ba9: baseline-arm64-clabbe 2026-03-21 05:39:03,421 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb8be9a1e2bfb1b3347ba9 2026-03-21 05:39:03,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:39:03,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb8be9a1e2bfb1b3347ba9 HTTP/1.1" 200 None 2026-03-21 05:39:03,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:39:03,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d510b8ad4124cec3566 HTTP/1.1" 200 None 2026-03-21 05:39:03,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:39:03,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb8be9a1e2bfb1b3347ba9 HTTP/1.1" 200 None 2026-03-21 05:39:03,482 - lava_callback - ERROR - Error processing callback for node 69bb8be9a1e2bfb1b3347ba9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47006 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47020 - "GET / HTTP/1.0" 200 OK 2026-03-21 05:39:21,506 - lava_callback - INFO - Received callback for node 69be2713d54b5122738c7211 INFO: 172.21.0.1:41098 - "POST /node/69be2713d54b5122738c7211 HTTP/1.0" 202 Accepted 2026-03-21 05:39:21,513 - lava_callback - INFO - Starting async job submit for node 69be2713d54b5122738c7211 2026-03-21 05:39:21,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:39:21,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be2713d54b5122738c7211 HTTP/1.1" 200 None 2026-03-21 05:39:21,547 - lava_callback - DEBUG - Processing node 69be2713d54b5122738c7211: baseline-arm64-broonie 2026-03-21 05:39:21,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 05:39:21,735 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 05:39:21,737 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69be2713d54b5122738c7211/log.txt.gz 2026-03-21 05:39:21,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 05:39:21,928 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 05:39:21,929 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69be2713d54b5122738c7211/lava-logs.yaml 2026-03-21 05:39:21,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 05:39:22,060 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 05:39:22,062 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69be2713d54b5122738c7211/lava_callback.json.gz 2026-03-21 05:39:22,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:39:22,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be2713d54b5122738c7211 HTTP/1.1" 200 None 2026-03-21 05:39:22,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:39:22,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-21 05:39:22,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:39:22,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be2713d54b5122738c7211 HTTP/1.1" 200 None 2026-03-21 05:39:22,222 - lava_callback - INFO - Completed processing callback for node 69be2713d54b5122738c7211 INFO: 172.21.0.1:41110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43442 - "GET / HTTP/1.0" 200 OK 2026-03-21 05:39:37,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:39:37,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:46906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34704 - "GET / HTTP/1.0" 200 OK 2026-03-21 05:40:13,107 - lava_callback - INFO - Received callback for node 69bb8c30a1e2bfb1b3347bb2 INFO: 172.21.0.1:40436 - "POST /node/69bb8c30a1e2bfb1b3347bb2 HTTP/1.0" 202 Accepted 2026-03-21 05:40:13,115 - lava_callback - INFO - Starting async job submit for node 69bb8c30a1e2bfb1b3347bb2 2026-03-21 05:40:13,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:40:13,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb8c30a1e2bfb1b3347bb2 HTTP/1.1" 200 None 2026-03-21 05:40:13,135 - lava_callback - DEBUG - Processing node 69bb8c30a1e2bfb1b3347bb2: baseline-arm64-baylibre 2026-03-21 05:40:13,135 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb8c30a1e2bfb1b3347bb2 2026-03-21 05:40:13,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:40:13,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb8c30a1e2bfb1b3347bb2 HTTP/1.1" 200 None 2026-03-21 05:40:13,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:40:13,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-21 05:40:13,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:40:13,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb8c30a1e2bfb1b3347bb2 HTTP/1.1" 200 None 2026-03-21 05:40:13,209 - lava_callback - ERROR - Error processing callback for node 69bb8c30a1e2bfb1b3347bb2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40438 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35406 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33160 - "GET / HTTP/1.0" 200 OK 2026-03-21 05:41:33,029 - lava_callback - INFO - Received callback for node 69bb8c7fa1e2bfb1b3347bc0 INFO: 172.21.0.1:59912 - "POST /node/69bb8c7fa1e2bfb1b3347bc0 HTTP/1.0" 202 Accepted 2026-03-21 05:41:33,035 - lava_callback - INFO - Starting async job submit for node 69bb8c7fa1e2bfb1b3347bc0 2026-03-21 05:41:33,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:41:33,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb8c7fa1e2bfb1b3347bc0 HTTP/1.1" 200 None 2026-03-21 05:41:33,051 - lava_callback - DEBUG - Processing node 69bb8c7fa1e2bfb1b3347bc0: baseline-arm64-baylibre 2026-03-21 05:41:33,051 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb8c7fa1e2bfb1b3347bc0 2026-03-21 05:41:33,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:41:33,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb8c7fa1e2bfb1b3347bc0 HTTP/1.1" 200 None 2026-03-21 05:41:33,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:41:33,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-21 05:41:33,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:41:33,105 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb8c7fa1e2bfb1b3347bc0 HTTP/1.1" 200 None 2026-03-21 05:41:33,106 - lava_callback - ERROR - Error processing callback for node 69bb8c7fa1e2bfb1b3347bc0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51032 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39756 - "GET / HTTP/1.0" 200 OK 2026-03-21 05:42:53,164 - lava_callback - INFO - Received callback for node 69bb8ccea1e2bfb1b3347bcb INFO: 172.21.0.1:49226 - "POST /node/69bb8ccea1e2bfb1b3347bcb HTTP/1.0" 202 Accepted 2026-03-21 05:42:53,170 - lava_callback - INFO - Starting async job submit for node 69bb8ccea1e2bfb1b3347bcb 2026-03-21 05:42:53,171 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:42:53,185 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb8ccea1e2bfb1b3347bcb HTTP/1.1" 200 None 2026-03-21 05:42:53,186 - lava_callback - DEBUG - Processing node 69bb8ccea1e2bfb1b3347bcb: baseline-arm64-baylibre 2026-03-21 05:42:53,186 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb8ccea1e2bfb1b3347bcb 2026-03-21 05:42:53,187 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:42:53,210 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb8ccea1e2bfb1b3347bcb HTTP/1.1" 200 None 2026-03-21 05:42:53,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:42:53,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-21 05:42:53,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:42:53,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb8ccea1e2bfb1b3347bcb HTTP/1.1" 200 None 2026-03-21 05:42:53,247 - lava_callback - ERROR - Error processing callback for node 69bb8ccea1e2bfb1b3347bcb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55352 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43636 - "GET / HTTP/1.0" 200 OK 2026-03-21 05:44:13,196 - lava_callback - INFO - Received callback for node 69bb8d1da1e2bfb1b3347bd7 INFO: 172.21.0.1:57360 - "POST /node/69bb8d1da1e2bfb1b3347bd7 HTTP/1.0" 202 Accepted 2026-03-21 05:44:13,205 - lava_callback - INFO - Starting async job submit for node 69bb8d1da1e2bfb1b3347bd7 2026-03-21 05:44:13,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:44:13,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb8d1da1e2bfb1b3347bd7 HTTP/1.1" 200 None 2026-03-21 05:44:13,223 - lava_callback - DEBUG - Processing node 69bb8d1da1e2bfb1b3347bd7: baseline-arm64-baylibre 2026-03-21 05:44:13,223 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb8d1da1e2bfb1b3347bd7 2026-03-21 05:44:13,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:44:13,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb8d1da1e2bfb1b3347bd7 HTTP/1.1" 200 None 2026-03-21 05:44:13,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:44:13,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-21 05:44:13,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:44:13,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb8d1da1e2bfb1b3347bd7 HTTP/1.1" 200 None 2026-03-21 05:44:13,276 - lava_callback - ERROR - Error processing callback for node 69bb8d1da1e2bfb1b3347bd7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57364 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49742 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34034 - "GET / HTTP/1.0" 200 OK 2026-03-21 05:45:33,302 - lava_callback - INFO - Received callback for node 69bb8d6ca1e2bfb1b3347be9 INFO: 172.21.0.1:46650 - "POST /node/69bb8d6ca1e2bfb1b3347be9 HTTP/1.0" 202 Accepted 2026-03-21 05:45:33,307 - lava_callback - INFO - Starting async job submit for node 69bb8d6ca1e2bfb1b3347be9 2026-03-21 05:45:33,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:45:33,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb8d6ca1e2bfb1b3347be9 HTTP/1.1" 200 None 2026-03-21 05:45:33,325 - lava_callback - DEBUG - Processing node 69bb8d6ca1e2bfb1b3347be9: baseline-arm64-baylibre 2026-03-21 05:45:33,325 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb8d6ca1e2bfb1b3347be9 2026-03-21 05:45:33,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:45:33,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb8d6ca1e2bfb1b3347be9 HTTP/1.1" 200 None 2026-03-21 05:45:33,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:45:33,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-21 05:45:33,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:45:33,374 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb8d6ca1e2bfb1b3347be9 HTTP/1.1" 200 None 2026-03-21 05:45:33,376 - lava_callback - ERROR - Error processing callback for node 69bb8d6ca1e2bfb1b3347be9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39762 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34110 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41068 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40956 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40966 - "GET / HTTP/1.0" 200 OK 2026-03-21 05:49:23,694 - lava_callback - INFO - Received callback for node 69bb8e5da1e2bfb1b3347c13 INFO: 172.21.0.1:40654 - "POST /node/69bb8e5da1e2bfb1b3347c13 HTTP/1.0" 202 Accepted 2026-03-21 05:49:23,700 - lava_callback - INFO - Starting async job submit for node 69bb8e5da1e2bfb1b3347c13 2026-03-21 05:49:23,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:49:23,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb8e5da1e2bfb1b3347c13 HTTP/1.1" 200 None 2026-03-21 05:49:23,728 - lava_callback - DEBUG - Processing node 69bb8e5da1e2bfb1b3347c13: baseline-arm64-clabbe 2026-03-21 05:49:23,728 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb8e5da1e2bfb1b3347c13 2026-03-21 05:49:23,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:49:23,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb8e5da1e2bfb1b3347c13 HTTP/1.1" 200 None 2026-03-21 05:49:23,743 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:49:23,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-21 05:49:23,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:49:23,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb8e5da1e2bfb1b3347c13 HTTP/1.1" 200 None 2026-03-21 05:49:23,781 - lava_callback - ERROR - Error processing callback for node 69bb8e5da1e2bfb1b3347c13: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51224 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40364 - "GET / HTTP/1.0" 200 OK 2026-03-21 05:50:33,563 - lava_callback - INFO - Received callback for node 69bb8ea6a1e2bfb1b3347c1a INFO: 172.21.0.1:44994 - "POST /node/69bb8ea6a1e2bfb1b3347c1a HTTP/1.0" 202 Accepted 2026-03-21 05:50:33,569 - lava_callback - INFO - Starting async job submit for node 69bb8ea6a1e2bfb1b3347c1a 2026-03-21 05:50:33,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:50:33,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb8ea6a1e2bfb1b3347c1a HTTP/1.1" 200 None 2026-03-21 05:50:33,588 - lava_callback - DEBUG - Processing node 69bb8ea6a1e2bfb1b3347c1a: baseline-arm64-baylibre 2026-03-21 05:50:33,589 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb8ea6a1e2bfb1b3347c1a 2026-03-21 05:50:33,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:50:33,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb8ea6a1e2bfb1b3347c1a HTTP/1.1" 200 None 2026-03-21 05:50:33,611 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:50:33,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d510b8ad4124cec3566 HTTP/1.1" 200 None 2026-03-21 05:50:33,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:50:33,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb8ea6a1e2bfb1b3347c1a HTTP/1.1" 200 None 2026-03-21 05:50:33,659 - lava_callback - ERROR - Error processing callback for node 69bb8ea6a1e2bfb1b3347c1a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59400 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44240 - "GET / HTTP/1.0" 200 OK 2026-03-21 05:52:03,816 - lava_callback - INFO - Received callback for node 69bb8efea1e2bfb1b3347c29 INFO: 172.21.0.1:45226 - "POST /node/69bb8efea1e2bfb1b3347c29 HTTP/1.0" 202 Accepted 2026-03-21 05:52:03,825 - lava_callback - INFO - Starting async job submit for node 69bb8efea1e2bfb1b3347c29 2026-03-21 05:52:03,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:52:03,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb8efea1e2bfb1b3347c29 HTTP/1.1" 200 None 2026-03-21 05:52:03,842 - lava_callback - DEBUG - Processing node 69bb8efea1e2bfb1b3347c29: baseline-arm64-clabbe 2026-03-21 05:52:03,842 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb8efea1e2bfb1b3347c29 2026-03-21 05:52:03,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:52:03,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb8efea1e2bfb1b3347c29 HTTP/1.1" 200 None 2026-03-21 05:52:03,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:52:03,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-21 05:52:03,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:52:03,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb8efea1e2bfb1b3347c29 HTTP/1.1" 200 None 2026-03-21 05:52:03,902 - lava_callback - ERROR - Error processing callback for node 69bb8efea1e2bfb1b3347c29: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59960 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39926 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39934 - "GET / HTTP/1.0" 200 OK 2026-03-21 05:53:23,881 - lava_callback - INFO - Received callback for node 69bb8f4ea1e2bfb1b3347c3c INFO: 172.21.0.1:54750 - "POST /node/69bb8f4ea1e2bfb1b3347c3c HTTP/1.0" 202 Accepted 2026-03-21 05:53:23,886 - lava_callback - INFO - Starting async job submit for node 69bb8f4ea1e2bfb1b3347c3c 2026-03-21 05:53:23,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:53:23,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb8f4ea1e2bfb1b3347c3c HTTP/1.1" 200 None 2026-03-21 05:53:23,903 - lava_callback - DEBUG - Processing node 69bb8f4ea1e2bfb1b3347c3c: baseline-arm64-clabbe 2026-03-21 05:53:23,904 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb8f4ea1e2bfb1b3347c3c 2026-03-21 05:53:23,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:53:23,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb8f4ea1e2bfb1b3347c3c HTTP/1.1" 200 None 2026-03-21 05:53:23,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:53:23,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-21 05:53:23,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:53:23,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb8f4ea1e2bfb1b3347c3c HTTP/1.1" 200 None 2026-03-21 05:53:23,971 - lava_callback - ERROR - Error processing callback for node 69bb8f4ea1e2bfb1b3347c3c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59564 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50150 - "GET / HTTP/1.0" 200 OK 2026-03-21 05:54:43,968 - lava_callback - INFO - Received callback for node 69bb8f9ea1e2bfb1b3347c42 INFO: 172.21.0.1:38204 - "POST /node/69bb8f9ea1e2bfb1b3347c42 HTTP/1.0" 202 Accepted 2026-03-21 05:54:43,973 - lava_callback - INFO - Starting async job submit for node 69bb8f9ea1e2bfb1b3347c42 2026-03-21 05:54:43,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:54:43,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb8f9ea1e2bfb1b3347c42 HTTP/1.1" 200 None 2026-03-21 05:54:43,989 - lava_callback - DEBUG - Processing node 69bb8f9ea1e2bfb1b3347c42: baseline-arm64-clabbe 2026-03-21 05:54:43,989 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb8f9ea1e2bfb1b3347c42 2026-03-21 05:54:43,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:54:44,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb8f9ea1e2bfb1b3347c42 HTTP/1.1" 200 None 2026-03-21 05:54:44,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:54:44,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-21 05:54:44,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:54:44,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb8f9ea1e2bfb1b3347c42 HTTP/1.1" 200 None 2026-03-21 05:54:44,050 - lava_callback - ERROR - Error processing callback for node 69bb8f9ea1e2bfb1b3347c42: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49798 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36144 - "GET / HTTP/1.0" 200 OK 2026-03-21 05:56:04,217 - lava_callback - INFO - Received callback for node 69bb8feea1e2bfb1b3347c51 INFO: 172.21.0.1:40072 - "POST /node/69bb8feea1e2bfb1b3347c51 HTTP/1.0" 202 Accepted 2026-03-21 05:56:04,222 - lava_callback - INFO - Starting async job submit for node 69bb8feea1e2bfb1b3347c51 2026-03-21 05:56:04,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:56:04,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb8feea1e2bfb1b3347c51 HTTP/1.1" 200 None 2026-03-21 05:56:04,238 - lava_callback - DEBUG - Processing node 69bb8feea1e2bfb1b3347c51: baseline-arm64-clabbe 2026-03-21 05:56:04,238 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb8feea1e2bfb1b3347c51 2026-03-21 05:56:04,239 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:56:04,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb8feea1e2bfb1b3347c51 HTTP/1.1" 200 None 2026-03-21 05:56:04,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:56:04,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-21 05:56:04,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:56:04,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb8feea1e2bfb1b3347c51 HTTP/1.1" 200 None 2026-03-21 05:56:04,328 - lava_callback - ERROR - Error processing callback for node 69bb8feea1e2bfb1b3347c51: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49382 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52870 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52882 - "GET / HTTP/1.0" 200 OK 2026-03-21 05:57:24,066 - lava_callback - INFO - Received callback for node 69bb903ca1e2bfb1b3347c64 INFO: 172.21.0.1:60678 - "POST /node/69bb903ca1e2bfb1b3347c64 HTTP/1.0" 202 Accepted 2026-03-21 05:57:24,071 - lava_callback - INFO - Starting async job submit for node 69bb903ca1e2bfb1b3347c64 2026-03-21 05:57:24,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:57:24,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb903ca1e2bfb1b3347c64 HTTP/1.1" 200 None 2026-03-21 05:57:24,088 - lava_callback - DEBUG - Processing node 69bb903ca1e2bfb1b3347c64: baseline-arm64-clabbe 2026-03-21 05:57:24,088 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb903ca1e2bfb1b3347c64 2026-03-21 05:57:24,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:57:24,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb903ca1e2bfb1b3347c64 HTTP/1.1" 200 None 2026-03-21 05:57:24,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:57:24,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-21 05:57:24,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:57:24,146 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb903ca1e2bfb1b3347c64 HTTP/1.1" 200 None 2026-03-21 05:57:24,147 - lava_callback - ERROR - Error processing callback for node 69bb903ca1e2bfb1b3347c64: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42998 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52326 - "GET / HTTP/1.0" 200 OK 2026-03-21 05:58:34,086 - lava_callback - INFO - Received callback for node 69bb9085a1e2bfb1b3347c6b INFO: 172.21.0.1:56938 - "POST /node/69bb9085a1e2bfb1b3347c6b HTTP/1.0" 202 Accepted 2026-03-21 05:58:34,092 - lava_callback - INFO - Starting async job submit for node 69bb9085a1e2bfb1b3347c6b 2026-03-21 05:58:34,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:58:34,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb9085a1e2bfb1b3347c6b HTTP/1.1" 200 None 2026-03-21 05:58:34,108 - lava_callback - DEBUG - Processing node 69bb9085a1e2bfb1b3347c6b: baseline-arm64-baylibre 2026-03-21 05:58:34,108 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb9085a1e2bfb1b3347c6b 2026-03-21 05:58:34,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:58:34,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb9085a1e2bfb1b3347c6b HTTP/1.1" 200 None 2026-03-21 05:58:34,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:58:34,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d510b8ad4124cec3566 HTTP/1.1" 200 None 2026-03-21 05:58:34,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:58:34,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb9085a1e2bfb1b3347c6b HTTP/1.1" 200 None 2026-03-21 05:58:34,162 - lava_callback - ERROR - Error processing callback for node 69bb9085a1e2bfb1b3347c6b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41474 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59114 - "GET / HTTP/1.0" 200 OK 2026-03-21 05:59:54,096 - lava_callback - INFO - Received callback for node 69bb90d4a1e2bfb1b3347c7a INFO: 172.21.0.1:32978 - "POST /node/69bb90d4a1e2bfb1b3347c7a HTTP/1.0" 202 Accepted 2026-03-21 05:59:54,102 - lava_callback - INFO - Starting async job submit for node 69bb90d4a1e2bfb1b3347c7a 2026-03-21 05:59:54,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:59:54,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb90d4a1e2bfb1b3347c7a HTTP/1.1" 200 None 2026-03-21 05:59:54,118 - lava_callback - DEBUG - Processing node 69bb90d4a1e2bfb1b3347c7a: baseline-arm64-baylibre 2026-03-21 05:59:54,118 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb90d4a1e2bfb1b3347c7a 2026-03-21 05:59:54,119 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:59:54,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb90d4a1e2bfb1b3347c7a HTTP/1.1" 200 None 2026-03-21 05:59:54,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:59:54,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d510b8ad4124cec3566 HTTP/1.1" 200 None 2026-03-21 05:59:54,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 05:59:54,187 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb90d4a1e2bfb1b3347c7a HTTP/1.1" 200 None 2026-03-21 05:59:54,189 - lava_callback - ERROR - Error processing callback for node 69bb90d4a1e2bfb1b3347c7a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:32980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45088 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47716 - "GET / HTTP/1.0" 200 OK 2026-03-21 06:01:14,166 - lava_callback - INFO - Received callback for node 69bb9126a1e2bfb1b3347c84 INFO: 172.21.0.1:48806 - "POST /node/69bb9126a1e2bfb1b3347c84 HTTP/1.0" 202 Accepted 2026-03-21 06:01:14,171 - lava_callback - INFO - Starting async job submit for node 69bb9126a1e2bfb1b3347c84 2026-03-21 06:01:14,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:01:14,187 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb9126a1e2bfb1b3347c84 HTTP/1.1" 200 None 2026-03-21 06:01:14,188 - lava_callback - DEBUG - Processing node 69bb9126a1e2bfb1b3347c84: baseline-arm64-baylibre 2026-03-21 06:01:14,188 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb9126a1e2bfb1b3347c84 2026-03-21 06:01:14,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:01:14,201 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb9126a1e2bfb1b3347c84 HTTP/1.1" 200 None 2026-03-21 06:01:14,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:01:14,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d510b8ad4124cec3566 HTTP/1.1" 200 None 2026-03-21 06:01:14,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:01:14,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb9126a1e2bfb1b3347c84 HTTP/1.1" 200 None 2026-03-21 06:01:14,261 - lava_callback - ERROR - Error processing callback for node 69bb9126a1e2bfb1b3347c84: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48818 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42996 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40332 - "GET / HTTP/1.0" 200 OK 2026-03-21 06:02:34,240 - lava_callback - INFO - Received callback for node 69bb9175a1e2bfb1b3347c92 INFO: 172.21.0.1:38642 - "POST /node/69bb9175a1e2bfb1b3347c92 HTTP/1.0" 202 Accepted 2026-03-21 06:02:34,246 - lava_callback - INFO - Starting async job submit for node 69bb9175a1e2bfb1b3347c92 2026-03-21 06:02:34,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:02:34,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb9175a1e2bfb1b3347c92 HTTP/1.1" 200 None 2026-03-21 06:02:34,261 - lava_callback - DEBUG - Processing node 69bb9175a1e2bfb1b3347c92: baseline-arm64-baylibre 2026-03-21 06:02:34,261 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb9175a1e2bfb1b3347c92 2026-03-21 06:02:34,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:02:34,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb9175a1e2bfb1b3347c92 HTTP/1.1" 200 None 2026-03-21 06:02:34,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:02:34,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d510b8ad4124cec3566 HTTP/1.1" 200 None 2026-03-21 06:02:34,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:02:34,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb9175a1e2bfb1b3347c92 HTTP/1.1" 200 None 2026-03-21 06:02:34,319 - lava_callback - ERROR - Error processing callback for node 69bb9175a1e2bfb1b3347c92: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58732 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37882 - "GET / HTTP/1.0" 200 OK 2026-03-21 06:03:54,346 - lava_callback - INFO - Received callback for node 69bb91c5a1e2bfb1b3347ca2 INFO: 172.21.0.1:40562 - "POST /node/69bb91c5a1e2bfb1b3347ca2 HTTP/1.0" 202 Accepted 2026-03-21 06:03:54,352 - lava_callback - INFO - Starting async job submit for node 69bb91c5a1e2bfb1b3347ca2 2026-03-21 06:03:54,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:03:54,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb91c5a1e2bfb1b3347ca2 HTTP/1.1" 200 None 2026-03-21 06:03:54,368 - lava_callback - DEBUG - Processing node 69bb91c5a1e2bfb1b3347ca2: baseline-arm64-baylibre 2026-03-21 06:03:54,368 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb91c5a1e2bfb1b3347ca2 2026-03-21 06:03:54,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:03:54,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb91c5a1e2bfb1b3347ca2 HTTP/1.1" 200 None 2026-03-21 06:03:54,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:03:54,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d510b8ad4124cec3566 HTTP/1.1" 200 None 2026-03-21 06:03:54,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:03:54,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb91c5a1e2bfb1b3347ca2 HTTP/1.1" 200 None 2026-03-21 06:03:54,431 - lava_callback - ERROR - Error processing callback for node 69bb91c5a1e2bfb1b3347ca2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49290 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45660 - "GET / HTTP/1.0" 200 OK 2026-03-21 06:05:14,340 - lava_callback - INFO - Received callback for node 69bb9216a1e2bfb1b3347cad INFO: 172.21.0.1:37044 - "POST /node/69bb9216a1e2bfb1b3347cad HTTP/1.0" 202 Accepted 2026-03-21 06:05:14,345 - lava_callback - INFO - Starting async job submit for node 69bb9216a1e2bfb1b3347cad 2026-03-21 06:05:14,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:05:14,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb9216a1e2bfb1b3347cad HTTP/1.1" 200 None 2026-03-21 06:05:14,368 - lava_callback - DEBUG - Processing node 69bb9216a1e2bfb1b3347cad: baseline-arm64-baylibre 2026-03-21 06:05:14,368 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb9216a1e2bfb1b3347cad 2026-03-21 06:05:14,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:05:14,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb9216a1e2bfb1b3347cad HTTP/1.1" 200 None 2026-03-21 06:05:14,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:05:14,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d510b8ad4124cec3566 HTTP/1.1" 200 None 2026-03-21 06:05:14,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:05:14,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb9216a1e2bfb1b3347cad HTTP/1.1" 200 None 2026-03-21 06:05:14,428 - lava_callback - ERROR - Error processing callback for node 69bb9216a1e2bfb1b3347cad: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37048 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58252 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32984 - "GET / HTTP/1.0" 200 OK 2026-03-21 06:06:44,474 - lava_callback - INFO - Received callback for node 69bb926ba1e2bfb1b3347cba INFO: 172.21.0.1:51052 - "POST /node/69bb926ba1e2bfb1b3347cba HTTP/1.0" 202 Accepted 2026-03-21 06:06:44,480 - lava_callback - INFO - Starting async job submit for node 69bb926ba1e2bfb1b3347cba 2026-03-21 06:06:44,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:06:44,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb926ba1e2bfb1b3347cba HTTP/1.1" 200 None 2026-03-21 06:06:44,496 - lava_callback - DEBUG - Processing node 69bb926ba1e2bfb1b3347cba: baseline-arm64-clabbe 2026-03-21 06:06:44,496 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb926ba1e2bfb1b3347cba 2026-03-21 06:06:44,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:06:44,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb926ba1e2bfb1b3347cba HTTP/1.1" 200 None 2026-03-21 06:06:44,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:06:44,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d510b8ad4124cec3566 HTTP/1.1" 200 None 2026-03-21 06:06:44,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:06:44,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb926ba1e2bfb1b3347cba HTTP/1.1" 200 None 2026-03-21 06:06:44,572 - lava_callback - ERROR - Error processing callback for node 69bb926ba1e2bfb1b3347cba: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47936 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48440 - "GET / HTTP/1.0" 200 OK 2026-03-21 06:08:04,606 - lava_callback - INFO - Received callback for node 69bb92baa1e2bfb1b3347cca INFO: 172.21.0.1:40646 - "POST /node/69bb92baa1e2bfb1b3347cca HTTP/1.0" 202 Accepted 2026-03-21 06:08:04,612 - lava_callback - INFO - Starting async job submit for node 69bb92baa1e2bfb1b3347cca 2026-03-21 06:08:04,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:08:04,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb92baa1e2bfb1b3347cca HTTP/1.1" 200 None 2026-03-21 06:08:04,628 - lava_callback - DEBUG - Processing node 69bb92baa1e2bfb1b3347cca: baseline-arm64-clabbe 2026-03-21 06:08:04,628 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb92baa1e2bfb1b3347cca 2026-03-21 06:08:04,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:08:04,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb92baa1e2bfb1b3347cca HTTP/1.1" 200 None 2026-03-21 06:08:04,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:08:04,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d510b8ad4124cec3566 HTTP/1.1" 200 None 2026-03-21 06:08:04,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:08:04,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb92baa1e2bfb1b3347cca HTTP/1.1" 200 None 2026-03-21 06:08:04,719 - lava_callback - ERROR - Error processing callback for node 69bb92baa1e2bfb1b3347cca: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44648 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58016 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58020 - "GET / HTTP/1.0" 200 OK 2026-03-21 06:09:24,591 - lava_callback - INFO - Received callback for node 69bb930ba1e2bfb1b3347cda INFO: 172.21.0.1:42440 - "POST /node/69bb930ba1e2bfb1b3347cda HTTP/1.0" 202 Accepted 2026-03-21 06:09:24,596 - lava_callback - INFO - Starting async job submit for node 69bb930ba1e2bfb1b3347cda 2026-03-21 06:09:24,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:09:24,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb930ba1e2bfb1b3347cda HTTP/1.1" 200 None 2026-03-21 06:09:24,613 - lava_callback - DEBUG - Processing node 69bb930ba1e2bfb1b3347cda: baseline-arm64-clabbe 2026-03-21 06:09:24,613 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb930ba1e2bfb1b3347cda 2026-03-21 06:09:24,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:09:24,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb930ba1e2bfb1b3347cda HTTP/1.1" 200 None 2026-03-21 06:09:24,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:09:24,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d510b8ad4124cec3566 HTTP/1.1" 200 None 2026-03-21 06:09:24,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:09:24,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb930ba1e2bfb1b3347cda HTTP/1.1" 200 None 2026-03-21 06:09:24,667 - lava_callback - ERROR - Error processing callback for node 69bb930ba1e2bfb1b3347cda: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53918 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57644 - "GET / HTTP/1.0" 200 OK 2026-03-21 06:10:44,611 - lava_callback - INFO - Received callback for node 69bb935ba1e2bfb1b3347cef INFO: 172.21.0.1:52426 - "POST /node/69bb935ba1e2bfb1b3347cef HTTP/1.0" 202 Accepted 2026-03-21 06:10:44,617 - lava_callback - INFO - Starting async job submit for node 69bb935ba1e2bfb1b3347cef 2026-03-21 06:10:44,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:10:44,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb935ba1e2bfb1b3347cef HTTP/1.1" 200 None 2026-03-21 06:10:44,638 - lava_callback - DEBUG - Processing node 69bb935ba1e2bfb1b3347cef: baseline-arm64-clabbe 2026-03-21 06:10:44,638 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb935ba1e2bfb1b3347cef 2026-03-21 06:10:44,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:10:44,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb935ba1e2bfb1b3347cef HTTP/1.1" 200 None 2026-03-21 06:10:44,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:10:44,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d510b8ad4124cec3566 HTTP/1.1" 200 None 2026-03-21 06:10:44,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:10:44,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb935ba1e2bfb1b3347cef HTTP/1.1" 200 None 2026-03-21 06:10:44,705 - lava_callback - ERROR - Error processing callback for node 69bb935ba1e2bfb1b3347cef: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55030 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48474 - "GET / HTTP/1.0" 200 OK 2026-03-21 06:12:04,698 - lava_callback - INFO - Received callback for node 69bb93afa1e2bfb1b3347d06 INFO: 172.21.0.1:40934 - "POST /node/69bb93afa1e2bfb1b3347d06 HTTP/1.0" 202 Accepted 2026-03-21 06:12:04,704 - lava_callback - INFO - Starting async job submit for node 69bb93afa1e2bfb1b3347d06 2026-03-21 06:12:04,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:12:04,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb93afa1e2bfb1b3347d06 HTTP/1.1" 200 None 2026-03-21 06:12:04,720 - lava_callback - DEBUG - Processing node 69bb93afa1e2bfb1b3347d06: baseline-arm64-clabbe 2026-03-21 06:12:04,720 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb93afa1e2bfb1b3347d06 2026-03-21 06:12:04,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:12:04,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb93afa1e2bfb1b3347d06 HTTP/1.1" 200 None 2026-03-21 06:12:04,743 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:12:04,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d510b8ad4124cec3566 HTTP/1.1" 200 None 2026-03-21 06:12:04,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:12:04,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb93afa1e2bfb1b3347d06 HTTP/1.1" 200 None 2026-03-21 06:12:04,787 - lava_callback - ERROR - Error processing callback for node 69bb93afa1e2bfb1b3347d06: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36258 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35476 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35484 - "GET / HTTP/1.0" 200 OK 2026-03-21 06:13:24,737 - lava_callback - INFO - Received callback for node 69bb93ffa1e2bfb1b3347d1c INFO: 172.21.0.1:50634 - "POST /node/69bb93ffa1e2bfb1b3347d1c HTTP/1.0" 202 Accepted 2026-03-21 06:13:24,743 - lava_callback - INFO - Starting async job submit for node 69bb93ffa1e2bfb1b3347d1c 2026-03-21 06:13:24,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:13:24,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb93ffa1e2bfb1b3347d1c HTTP/1.1" 200 None 2026-03-21 06:13:24,759 - lava_callback - DEBUG - Processing node 69bb93ffa1e2bfb1b3347d1c: baseline-arm64-clabbe 2026-03-21 06:13:24,759 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb93ffa1e2bfb1b3347d1c 2026-03-21 06:13:24,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:13:24,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb93ffa1e2bfb1b3347d1c HTTP/1.1" 200 None 2026-03-21 06:13:24,779 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:13:24,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d510b8ad4124cec3566 HTTP/1.1" 200 None 2026-03-21 06:13:24,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:13:24,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb93ffa1e2bfb1b3347d1c HTTP/1.1" 200 None 2026-03-21 06:13:24,853 - lava_callback - ERROR - Error processing callback for node 69bb93ffa1e2bfb1b3347d1c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52852 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.21.0.1:52856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48078 - "GET /.well-known/security.txt HTTP/1.0" 404 Not Found INFO: 172.21.0.1:48082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51660 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60672 - "GET / HTTP/1.0" 200 OK 2026-03-21 06:15:04,852 - lava_callback - INFO - Received callback for node 69bb9453a1e2bfb1b3347d2a INFO: 172.21.0.1:57600 - "POST /node/69bb9453a1e2bfb1b3347d2a HTTP/1.0" 202 Accepted 2026-03-21 06:15:04,858 - lava_callback - INFO - Starting async job submit for node 69bb9453a1e2bfb1b3347d2a 2026-03-21 06:15:04,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:15:04,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb9453a1e2bfb1b3347d2a HTTP/1.1" 200 None 2026-03-21 06:15:04,874 - lava_callback - DEBUG - Processing node 69bb9453a1e2bfb1b3347d2a: baseline-arm64-clabbe 2026-03-21 06:15:04,875 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb9453a1e2bfb1b3347d2a 2026-03-21 06:15:04,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:15:04,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb9453a1e2bfb1b3347d2a HTTP/1.1" 200 None 2026-03-21 06:15:04,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:15:04,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d510b8ad4124cec3566 HTTP/1.1" 200 None 2026-03-21 06:15:04,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:15:04,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb9453a1e2bfb1b3347d2a HTTP/1.1" 200 None 2026-03-21 06:15:04,926 - lava_callback - ERROR - Error processing callback for node 69bb9453a1e2bfb1b3347d2a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49226 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42316 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60350 - "GET / HTTP/1.0" 200 OK 2026-03-21 06:16:26,537 - lava_callback - INFO - Received callback for node 69bb94a3a1e2bfb1b3347d30 INFO: 172.21.0.1:60358 - "POST /node/69bb94a3a1e2bfb1b3347d30 HTTP/1.0" 202 Accepted 2026-03-21 06:16:26,543 - lava_callback - INFO - Starting async job submit for node 69bb94a3a1e2bfb1b3347d30 2026-03-21 06:16:26,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:16:26,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb94a3a1e2bfb1b3347d30 HTTP/1.1" 200 None 2026-03-21 06:16:26,559 - lava_callback - DEBUG - Processing node 69bb94a3a1e2bfb1b3347d30: baseline-arm64-clabbe 2026-03-21 06:16:26,559 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb94a3a1e2bfb1b3347d30 2026-03-21 06:16:26,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:16:26,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb94a3a1e2bfb1b3347d30 HTTP/1.1" 200 None 2026-03-21 06:16:26,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:16:26,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d510b8ad4124cec3566 HTTP/1.1" 200 None 2026-03-21 06:16:26,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:16:26,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb94a3a1e2bfb1b3347d30 HTTP/1.1" 200 None 2026-03-21 06:16:26,658 - lava_callback - ERROR - Error processing callback for node 69bb94a3a1e2bfb1b3347d30: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57560 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57576 - "GET / HTTP/1.0" 200 OK 2026-03-21 06:17:24,887 - lava_callback - INFO - Received callback for node 69bb94f2a1e2bfb1b3347d3d INFO: 172.21.0.1:54626 - "POST /node/69bb94f2a1e2bfb1b3347d3d HTTP/1.0" 202 Accepted 2026-03-21 06:17:24,892 - lava_callback - INFO - Starting async job submit for node 69bb94f2a1e2bfb1b3347d3d 2026-03-21 06:17:24,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:17:24,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb94f2a1e2bfb1b3347d3d HTTP/1.1" 200 None 2026-03-21 06:17:24,907 - lava_callback - DEBUG - Processing node 69bb94f2a1e2bfb1b3347d3d: baseline-arm64-clabbe 2026-03-21 06:17:24,908 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb94f2a1e2bfb1b3347d3d 2026-03-21 06:17:24,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:17:24,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb94f2a1e2bfb1b3347d3d HTTP/1.1" 200 None 2026-03-21 06:17:24,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:17:24,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d510b8ad4124cec3566 HTTP/1.1" 200 None 2026-03-21 06:17:24,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:17:24,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb94f2a1e2bfb1b3347d3d HTTP/1.1" 200 None 2026-03-21 06:17:24,975 - lava_callback - ERROR - Error processing callback for node 69bb94f2a1e2bfb1b3347d3d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34218 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48156 - "GET / HTTP/1.0" 200 OK 2026-03-21 06:19:05,014 - lava_callback - INFO - Received callback for node 69bb9542a1e2bfb1b3347d4d INFO: 172.21.0.1:44434 - "POST /node/69bb9542a1e2bfb1b3347d4d HTTP/1.0" 202 Accepted 2026-03-21 06:19:05,020 - lava_callback - INFO - Starting async job submit for node 69bb9542a1e2bfb1b3347d4d 2026-03-21 06:19:05,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:19:05,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb9542a1e2bfb1b3347d4d HTTP/1.1" 200 None 2026-03-21 06:19:05,055 - lava_callback - DEBUG - Processing node 69bb9542a1e2bfb1b3347d4d: baseline-arm64-clabbe 2026-03-21 06:19:05,055 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb9542a1e2bfb1b3347d4d 2026-03-21 06:19:05,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:19:05,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb9542a1e2bfb1b3347d4d HTTP/1.1" 200 None 2026-03-21 06:19:05,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:19:05,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d510b8ad4124cec3566 HTTP/1.1" 200 None 2026-03-21 06:19:05,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:19:05,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb9542a1e2bfb1b3347d4d HTTP/1.1" 200 None 2026-03-21 06:19:05,108 - lava_callback - ERROR - Error processing callback for node 69bb9542a1e2bfb1b3347d4d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49286 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53458 - "GET / HTTP/1.0" 200 OK 2026-03-21 06:20:05,178 - lava_callback - INFO - Received callback for node 69bb9591a1e2bfb1b3347d59 INFO: 172.21.0.1:47666 - "POST /node/69bb9591a1e2bfb1b3347d59 HTTP/1.0" 202 Accepted 2026-03-21 06:20:05,183 - lava_callback - INFO - Starting async job submit for node 69bb9591a1e2bfb1b3347d59 2026-03-21 06:20:05,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:20:05,220 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb9591a1e2bfb1b3347d59 HTTP/1.1" 200 None 2026-03-21 06:20:05,221 - lava_callback - DEBUG - Processing node 69bb9591a1e2bfb1b3347d59: baseline-arm64-clabbe 2026-03-21 06:20:05,221 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb9591a1e2bfb1b3347d59 2026-03-21 06:20:05,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:20:05,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb9591a1e2bfb1b3347d59 HTTP/1.1" 200 None 2026-03-21 06:20:05,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:20:05,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d510b8ad4124cec3566 HTTP/1.1" 200 None 2026-03-21 06:20:05,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:20:05,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb9591a1e2bfb1b3347d59 HTTP/1.1" 200 None 2026-03-21 06:20:05,304 - lava_callback - ERROR - Error processing callback for node 69bb9591a1e2bfb1b3347d59: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40206 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58886 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58890 - "GET / HTTP/1.0" 200 OK 2026-03-21 06:21:25,122 - lava_callback - INFO - Received callback for node 69bb95e1a1e2bfb1b3347d67 INFO: 172.21.0.1:48448 - "POST /node/69bb95e1a1e2bfb1b3347d67 HTTP/1.0" 202 Accepted 2026-03-21 06:21:25,128 - lava_callback - INFO - Starting async job submit for node 69bb95e1a1e2bfb1b3347d67 2026-03-21 06:21:25,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:21:25,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb95e1a1e2bfb1b3347d67 HTTP/1.1" 200 None 2026-03-21 06:21:25,156 - lava_callback - DEBUG - Processing node 69bb95e1a1e2bfb1b3347d67: baseline-arm64-clabbe 2026-03-21 06:21:25,156 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb95e1a1e2bfb1b3347d67 2026-03-21 06:21:25,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:21:25,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb95e1a1e2bfb1b3347d67 HTTP/1.1" 200 None 2026-03-21 06:21:25,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:21:25,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d510b8ad4124cec3566 HTTP/1.1" 200 None 2026-03-21 06:21:25,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:21:25,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb95e1a1e2bfb1b3347d67 HTTP/1.1" 200 None 2026-03-21 06:21:25,218 - lava_callback - ERROR - Error processing callback for node 69bb95e1a1e2bfb1b3347d67: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43624 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47344 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59222 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35550 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50278 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56372 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47108 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50418 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52778 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38782 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37668 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52096 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50204 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53774 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59580 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54946 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34866 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54610 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53562 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44200 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39804 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42372 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54046 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46754 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32840 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:32854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39830 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58186 - "GET / HTTP/1.0" 200 OK 2026-03-21 06:47:36,916 - lava_callback - INFO - Received callback for node 69bb9bfca1e2bfb1b3347db1 INFO: 172.21.0.1:58192 - "POST /node/69bb9bfca1e2bfb1b3347db1 HTTP/1.0" 202 Accepted 2026-03-21 06:47:36,921 - lava_callback - INFO - Starting async job submit for node 69bb9bfca1e2bfb1b3347db1 2026-03-21 06:47:36,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:47:36,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb9bfca1e2bfb1b3347db1 HTTP/1.1" 200 None 2026-03-21 06:47:36,962 - lava_callback - DEBUG - Processing node 69bb9bfca1e2bfb1b3347db1: baseline-arm-baylibre 2026-03-21 06:47:36,962 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb9bfca1e2bfb1b3347db1 2026-03-21 06:47:36,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:47:36,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb9bfca1e2bfb1b3347db1 HTTP/1.1" 200 None 2026-03-21 06:47:36,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:47:36,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-21 06:47:37,000 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:47:37,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb9bfca1e2bfb1b3347db1 HTTP/1.1" 200 None 2026-03-21 06:47:37,058 - lava_callback - ERROR - Error processing callback for node 69bb9bfca1e2bfb1b3347db1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44476 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44490 - "GET / HTTP/1.0" 200 OK 2026-03-21 06:48:16,711 - lava_callback - INFO - Received callback for node 69bb9c20a1e2bfb1b3347dc7 INFO: 172.21.0.1:44502 - "POST /node/69bb9c20a1e2bfb1b3347dc7 HTTP/1.0" 202 Accepted 2026-03-21 06:48:16,717 - lava_callback - INFO - Starting async job submit for node 69bb9c20a1e2bfb1b3347dc7 2026-03-21 06:48:16,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:48:16,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb9c20a1e2bfb1b3347dc7 HTTP/1.1" 200 None 2026-03-21 06:48:16,732 - lava_callback - DEBUG - Processing node 69bb9c20a1e2bfb1b3347dc7: baseline-arm-baylibre 2026-03-21 06:48:16,732 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb9c20a1e2bfb1b3347dc7 2026-03-21 06:48:16,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:48:16,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb9c20a1e2bfb1b3347dc7 HTTP/1.1" 200 None 2026-03-21 06:48:16,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:48:16,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-21 06:48:16,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:48:16,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb9c20a1e2bfb1b3347dc7 HTTP/1.1" 200 None 2026-03-21 06:48:16,803 - lava_callback - ERROR - Error processing callback for node 69bb9c20a1e2bfb1b3347dc7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53420 - "GET / HTTP/1.0" 200 OK 2026-03-21 06:48:56,760 - lava_callback - INFO - Received callback for node 69bb9c45a1e2bfb1b3347dd1 INFO: 172.21.0.1:53424 - "POST /node/69bb9c45a1e2bfb1b3347dd1 HTTP/1.0" 202 Accepted 2026-03-21 06:48:56,765 - lava_callback - INFO - Starting async job submit for node 69bb9c45a1e2bfb1b3347dd1 2026-03-21 06:48:56,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:48:56,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb9c45a1e2bfb1b3347dd1 HTTP/1.1" 200 None 2026-03-21 06:48:56,789 - lava_callback - DEBUG - Processing node 69bb9c45a1e2bfb1b3347dd1: baseline-arm-baylibre 2026-03-21 06:48:56,789 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb9c45a1e2bfb1b3347dd1 2026-03-21 06:48:56,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:48:56,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb9c45a1e2bfb1b3347dd1 HTTP/1.1" 200 None 2026-03-21 06:48:56,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:48:56,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-21 06:48:56,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:48:56,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb9c45a1e2bfb1b3347dd1 HTTP/1.1" 200 None 2026-03-21 06:48:56,849 - lava_callback - ERROR - Error processing callback for node 69bb9c45a1e2bfb1b3347dd1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49742 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49748 - "GET / HTTP/1.0" 200 OK 2026-03-21 06:49:16,756 - lava_callback - INFO - Received callback for node 69bb9c69a1e2bfb1b3347dde INFO: 172.21.0.1:49760 - "POST /node/69bb9c69a1e2bfb1b3347dde HTTP/1.0" 202 Accepted 2026-03-21 06:49:16,761 - lava_callback - INFO - Starting async job submit for node 69bb9c69a1e2bfb1b3347dde 2026-03-21 06:49:16,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:49:16,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb9c69a1e2bfb1b3347dde HTTP/1.1" 200 None 2026-03-21 06:49:16,778 - lava_callback - DEBUG - Processing node 69bb9c69a1e2bfb1b3347dde: baseline-arm-baylibre 2026-03-21 06:49:16,778 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb9c69a1e2bfb1b3347dde 2026-03-21 06:49:16,779 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:49:16,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb9c69a1e2bfb1b3347dde HTTP/1.1" 200 None 2026-03-21 06:49:16,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:49:16,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-21 06:49:16,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:49:16,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb9c69a1e2bfb1b3347dde HTTP/1.1" 200 None 2026-03-21 06:49:16,830 - lava_callback - ERROR - Error processing callback for node 69bb9c69a1e2bfb1b3347dde: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36756 - "GET / HTTP/1.0" 200 OK 2026-03-21 06:49:56,775 - lava_callback - INFO - Received callback for node 69bb9c8ea1e2bfb1b3347de6 INFO: 172.21.0.1:36770 - "POST /node/69bb9c8ea1e2bfb1b3347de6 HTTP/1.0" 202 Accepted 2026-03-21 06:49:56,781 - lava_callback - INFO - Starting async job submit for node 69bb9c8ea1e2bfb1b3347de6 2026-03-21 06:49:56,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:49:56,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb9c8ea1e2bfb1b3347de6 HTTP/1.1" 200 None 2026-03-21 06:49:56,797 - lava_callback - DEBUG - Processing node 69bb9c8ea1e2bfb1b3347de6: baseline-arm-baylibre 2026-03-21 06:49:56,797 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb9c8ea1e2bfb1b3347de6 2026-03-21 06:49:56,798 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:49:56,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb9c8ea1e2bfb1b3347de6 HTTP/1.1" 200 None 2026-03-21 06:49:56,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:49:56,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-21 06:49:56,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:49:56,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb9c8ea1e2bfb1b3347de6 HTTP/1.1" 200 None 2026-03-21 06:49:56,857 - lava_callback - ERROR - Error processing callback for node 69bb9c8ea1e2bfb1b3347de6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46668 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49286 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38824 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38828 - "GET / HTTP/1.0" 200 OK 2026-03-21 06:52:26,307 - lava_callback - INFO - Received callback for node 69bb9d1ea1e2bfb1b3347e17 INFO: 172.21.0.1:33450 - "POST /node/69bb9d1ea1e2bfb1b3347e17 HTTP/1.0" 202 Accepted 2026-03-21 06:52:26,313 - lava_callback - INFO - Starting async job submit for node 69bb9d1ea1e2bfb1b3347e17 2026-03-21 06:52:26,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:52:26,327 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb9d1ea1e2bfb1b3347e17 HTTP/1.1" 200 None 2026-03-21 06:52:26,328 - lava_callback - DEBUG - Processing node 69bb9d1ea1e2bfb1b3347e17: baseline-arm-clabbe 2026-03-21 06:52:26,328 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb9d1ea1e2bfb1b3347e17 2026-03-21 06:52:26,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:52:26,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb9d1ea1e2bfb1b3347e17 HTTP/1.1" 200 None 2026-03-21 06:52:26,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:52:26,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-21 06:52:26,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:52:26,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb9d1ea1e2bfb1b3347e17 HTTP/1.1" 200 None 2026-03-21 06:52:26,388 - lava_callback - ERROR - Error processing callback for node 69bb9d1ea1e2bfb1b3347e17: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58590 - "GET / HTTP/1.0" 200 OK 2026-03-21 06:53:06,379 - lava_callback - INFO - Received callback for node 69bb9d43a1e2bfb1b3347e23 INFO: 172.21.0.1:43286 - "POST /node/69bb9d43a1e2bfb1b3347e23 HTTP/1.0" 202 Accepted 2026-03-21 06:53:06,385 - lava_callback - INFO - Starting async job submit for node 69bb9d43a1e2bfb1b3347e23 2026-03-21 06:53:06,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:53:06,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb9d43a1e2bfb1b3347e23 HTTP/1.1" 200 None 2026-03-21 06:53:06,401 - lava_callback - DEBUG - Processing node 69bb9d43a1e2bfb1b3347e23: baseline-arm-clabbe 2026-03-21 06:53:06,401 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb9d43a1e2bfb1b3347e23 2026-03-21 06:53:06,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:53:06,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb9d43a1e2bfb1b3347e23 HTTP/1.1" 200 None 2026-03-21 06:53:06,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:53:06,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-21 06:53:06,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:53:06,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb9d43a1e2bfb1b3347e23 HTTP/1.1" 200 None 2026-03-21 06:53:06,454 - lava_callback - ERROR - Error processing callback for node 69bb9d43a1e2bfb1b3347e23: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42540 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46916 - "GET / HTTP/1.0" 200 OK 2026-03-21 06:53:46,359 - lava_callback - INFO - Received callback for node 69bb9d68a1e2bfb1b3347e30 INFO: 172.21.0.1:38584 - "POST /node/69bb9d68a1e2bfb1b3347e30 HTTP/1.0" 202 Accepted 2026-03-21 06:53:46,364 - lava_callback - INFO - Starting async job submit for node 69bb9d68a1e2bfb1b3347e30 2026-03-21 06:53:46,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:53:46,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb9d68a1e2bfb1b3347e30 HTTP/1.1" 200 None 2026-03-21 06:53:46,394 - lava_callback - DEBUG - Processing node 69bb9d68a1e2bfb1b3347e30: baseline-arm-clabbe 2026-03-21 06:53:46,394 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb9d68a1e2bfb1b3347e30 2026-03-21 06:53:46,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:53:46,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb9d68a1e2bfb1b3347e30 HTTP/1.1" 200 None 2026-03-21 06:53:46,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:53:46,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-21 06:53:46,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:53:46,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb9d68a1e2bfb1b3347e30 HTTP/1.1" 200 None 2026-03-21 06:53:46,460 - lava_callback - ERROR - Error processing callback for node 69bb9d68a1e2bfb1b3347e30: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42256 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42270 - "GET / HTTP/1.0" 200 OK 2026-03-21 06:54:26,338 - lava_callback - INFO - Received callback for node 69bb9d8da1e2bfb1b3347e3d INFO: 172.21.0.1:54418 - "POST /node/69bb9d8da1e2bfb1b3347e3d HTTP/1.0" 202 Accepted 2026-03-21 06:54:26,343 - lava_callback - INFO - Starting async job submit for node 69bb9d8da1e2bfb1b3347e3d 2026-03-21 06:54:26,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:54:26,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb9d8da1e2bfb1b3347e3d HTTP/1.1" 200 None 2026-03-21 06:54:26,360 - lava_callback - DEBUG - Processing node 69bb9d8da1e2bfb1b3347e3d: baseline-arm-clabbe 2026-03-21 06:54:26,361 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb9d8da1e2bfb1b3347e3d 2026-03-21 06:54:26,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:54:26,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb9d8da1e2bfb1b3347e3d HTTP/1.1" 200 None 2026-03-21 06:54:26,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:54:26,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-21 06:54:26,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:54:26,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb9d8da1e2bfb1b3347e3d HTTP/1.1" 200 None 2026-03-21 06:54:26,419 - lava_callback - ERROR - Error processing callback for node 69bb9d8da1e2bfb1b3347e3d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52672 - "GET / HTTP/1.0" 200 OK 2026-03-21 06:54:46,395 - lava_callback - INFO - Received callback for node 69bb9db1a1e2bfb1b3347e4b INFO: 172.21.0.1:47714 - "POST /node/69bb9db1a1e2bfb1b3347e4b HTTP/1.0" 202 Accepted 2026-03-21 06:54:46,400 - lava_callback - INFO - Starting async job submit for node 69bb9db1a1e2bfb1b3347e4b 2026-03-21 06:54:46,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:54:46,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb9db1a1e2bfb1b3347e4b HTTP/1.1" 200 None 2026-03-21 06:54:46,419 - lava_callback - DEBUG - Processing node 69bb9db1a1e2bfb1b3347e4b: baseline-arm-clabbe 2026-03-21 06:54:46,419 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb9db1a1e2bfb1b3347e4b 2026-03-21 06:54:46,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:54:46,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb9db1a1e2bfb1b3347e4b HTTP/1.1" 200 None 2026-03-21 06:54:46,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:54:46,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-21 06:54:46,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:54:46,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb9db1a1e2bfb1b3347e4b HTTP/1.1" 200 None 2026-03-21 06:54:46,472 - lava_callback - ERROR - Error processing callback for node 69bb9db1a1e2bfb1b3347e4b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35752 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35768 - "GET / HTTP/1.0" 200 OK 2026-03-21 06:55:17,110 - lava_callback - INFO - Received callback for node 69bb9dd1a1e2bfb1b3347e58 INFO: 172.21.0.1:35780 - "POST /node/69bb9dd1a1e2bfb1b3347e58 HTTP/1.0" 202 Accepted 2026-03-21 06:55:17,116 - lava_callback - INFO - Starting async job submit for node 69bb9dd1a1e2bfb1b3347e58 2026-03-21 06:55:17,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:55:17,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb9dd1a1e2bfb1b3347e58 HTTP/1.1" 200 None 2026-03-21 06:55:17,132 - lava_callback - DEBUG - Processing node 69bb9dd1a1e2bfb1b3347e58: baseline-arm-baylibre 2026-03-21 06:55:17,132 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb9dd1a1e2bfb1b3347e58 2026-03-21 06:55:17,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:55:17,146 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb9dd1a1e2bfb1b3347e58 HTTP/1.1" 200 None 2026-03-21 06:55:17,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:55:17,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89da7fbe29c53ae85cce2 HTTP/1.1" 200 None 2026-03-21 06:55:17,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:55:17,187 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb9dd1a1e2bfb1b3347e58 HTTP/1.1" 200 None 2026-03-21 06:55:17,192 - lava_callback - ERROR - Error processing callback for node 69bb9dd1a1e2bfb1b3347e58: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41974 - "GET / HTTP/1.0" 200 OK 2026-03-21 06:55:57,095 - lava_callback - INFO - Received callback for node 69bb9df5a1e2bfb1b3347e69 INFO: 172.21.0.1:41986 - "POST /node/69bb9df5a1e2bfb1b3347e69 HTTP/1.0" 202 Accepted 2026-03-21 06:55:57,100 - lava_callback - INFO - Starting async job submit for node 69bb9df5a1e2bfb1b3347e69 2026-03-21 06:55:57,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:55:57,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb9df5a1e2bfb1b3347e69 HTTP/1.1" 200 None 2026-03-21 06:55:57,118 - lava_callback - DEBUG - Processing node 69bb9df5a1e2bfb1b3347e69: baseline-arm-baylibre 2026-03-21 06:55:57,118 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb9df5a1e2bfb1b3347e69 2026-03-21 06:55:57,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:55:57,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb9df5a1e2bfb1b3347e69 HTTP/1.1" 200 None 2026-03-21 06:55:57,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:55:57,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89da7fbe29c53ae85cce2 HTTP/1.1" 200 None 2026-03-21 06:55:57,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:55:57,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb9df5a1e2bfb1b3347e69 HTTP/1.1" 200 None 2026-03-21 06:55:57,180 - lava_callback - ERROR - Error processing callback for node 69bb9df5a1e2bfb1b3347e69: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39902 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45592 - "GET / HTTP/1.0" 200 OK 2026-03-21 06:56:37,235 - lava_callback - INFO - Received callback for node 69bb9e1aa1e2bfb1b3347e76 INFO: 172.21.0.1:45594 - "POST /node/69bb9e1aa1e2bfb1b3347e76 HTTP/1.0" 202 Accepted 2026-03-21 06:56:37,240 - lava_callback - INFO - Starting async job submit for node 69bb9e1aa1e2bfb1b3347e76 2026-03-21 06:56:37,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:56:37,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb9e1aa1e2bfb1b3347e76 HTTP/1.1" 200 None 2026-03-21 06:56:37,256 - lava_callback - DEBUG - Processing node 69bb9e1aa1e2bfb1b3347e76: baseline-arm-baylibre 2026-03-21 06:56:37,256 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb9e1aa1e2bfb1b3347e76 2026-03-21 06:56:37,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:56:37,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb9e1aa1e2bfb1b3347e76 HTTP/1.1" 200 None 2026-03-21 06:56:37,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:56:37,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89da7fbe29c53ae85cce2 HTTP/1.1" 200 None 2026-03-21 06:56:37,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:56:37,307 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb9e1aa1e2bfb1b3347e76 HTTP/1.1" 200 None 2026-03-21 06:56:37,308 - lava_callback - ERROR - Error processing callback for node 69bb9e1aa1e2bfb1b3347e76: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44080 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44092 - "GET / HTTP/1.0" 200 OK 2026-03-21 06:57:17,161 - lava_callback - INFO - Received callback for node 69bb9e3ea1e2bfb1b3347e82 INFO: 172.21.0.1:44098 - "POST /node/69bb9e3ea1e2bfb1b3347e82 HTTP/1.0" 202 Accepted 2026-03-21 06:57:17,166 - lava_callback - INFO - Starting async job submit for node 69bb9e3ea1e2bfb1b3347e82 2026-03-21 06:57:17,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:57:17,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb9e3ea1e2bfb1b3347e82 HTTP/1.1" 200 None 2026-03-21 06:57:17,183 - lava_callback - DEBUG - Processing node 69bb9e3ea1e2bfb1b3347e82: baseline-arm-baylibre 2026-03-21 06:57:17,183 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb9e3ea1e2bfb1b3347e82 2026-03-21 06:57:17,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:57:17,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb9e3ea1e2bfb1b3347e82 HTTP/1.1" 200 None 2026-03-21 06:57:17,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:57:17,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89da7fbe29c53ae85cce2 HTTP/1.1" 200 None 2026-03-21 06:57:17,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:57:17,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb9e3ea1e2bfb1b3347e82 HTTP/1.1" 200 None 2026-03-21 06:57:17,257 - lava_callback - ERROR - Error processing callback for node 69bb9e3ea1e2bfb1b3347e82: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49212 - "GET / HTTP/1.0" 200 OK 2026-03-21 06:57:57,314 - lava_callback - INFO - Received callback for node 69bb9e62a1e2bfb1b3347e8f INFO: 172.21.0.1:49224 - "POST /node/69bb9e62a1e2bfb1b3347e8f HTTP/1.0" 202 Accepted 2026-03-21 06:57:57,320 - lava_callback - INFO - Starting async job submit for node 69bb9e62a1e2bfb1b3347e8f 2026-03-21 06:57:57,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:57:57,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb9e62a1e2bfb1b3347e8f HTTP/1.1" 200 None 2026-03-21 06:57:57,335 - lava_callback - DEBUG - Processing node 69bb9e62a1e2bfb1b3347e8f: baseline-arm-baylibre 2026-03-21 06:57:57,336 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb9e62a1e2bfb1b3347e8f 2026-03-21 06:57:57,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:57:57,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb9e62a1e2bfb1b3347e8f HTTP/1.1" 200 None 2026-03-21 06:57:57,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:57:57,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89da7fbe29c53ae85cce2 HTTP/1.1" 200 None 2026-03-21 06:57:57,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:57:57,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb9e62a1e2bfb1b3347e8f HTTP/1.1" 200 None 2026-03-21 06:57:57,391 - lava_callback - ERROR - Error processing callback for node 69bb9e62a1e2bfb1b3347e8f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45114 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45128 - "GET / HTTP/1.0" 200 OK 2026-03-21 06:58:17,208 - lava_callback - INFO - Received callback for node 69bb9e87a1e2bfb1b3347e9f INFO: 172.21.0.1:45130 - "POST /node/69bb9e87a1e2bfb1b3347e9f HTTP/1.0" 202 Accepted 2026-03-21 06:58:17,214 - lava_callback - INFO - Starting async job submit for node 69bb9e87a1e2bfb1b3347e9f 2026-03-21 06:58:17,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:58:17,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb9e87a1e2bfb1b3347e9f HTTP/1.1" 200 None 2026-03-21 06:58:17,234 - lava_callback - DEBUG - Processing node 69bb9e87a1e2bfb1b3347e9f: baseline-arm-baylibre 2026-03-21 06:58:17,234 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb9e87a1e2bfb1b3347e9f 2026-03-21 06:58:17,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:58:17,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb9e87a1e2bfb1b3347e9f HTTP/1.1" 200 None 2026-03-21 06:58:17,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:58:17,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89da7fbe29c53ae85cce2 HTTP/1.1" 200 None 2026-03-21 06:58:17,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:58:17,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb9e87a1e2bfb1b3347e9f HTTP/1.1" 200 None 2026-03-21 06:58:17,292 - lava_callback - ERROR - Error processing callback for node 69bb9e87a1e2bfb1b3347e9f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46424 - "GET / HTTP/1.0" 200 OK 2026-03-21 06:58:57,257 - lava_callback - INFO - Received callback for node 69bb9ea7a1e2bfb1b3347ead INFO: 172.21.0.1:46430 - "POST /node/69bb9ea7a1e2bfb1b3347ead HTTP/1.0" 202 Accepted 2026-03-21 06:58:57,262 - lava_callback - INFO - Starting async job submit for node 69bb9ea7a1e2bfb1b3347ead 2026-03-21 06:58:57,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:58:57,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb9ea7a1e2bfb1b3347ead HTTP/1.1" 200 None 2026-03-21 06:58:57,279 - lava_callback - DEBUG - Processing node 69bb9ea7a1e2bfb1b3347ead: baseline-arm-baylibre 2026-03-21 06:58:57,279 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb9ea7a1e2bfb1b3347ead 2026-03-21 06:58:57,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:58:57,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb9ea7a1e2bfb1b3347ead HTTP/1.1" 200 None 2026-03-21 06:58:57,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:58:57,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89da7fbe29c53ae85cce2 HTTP/1.1" 200 None 2026-03-21 06:58:57,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:58:57,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb9ea7a1e2bfb1b3347ead HTTP/1.1" 200 None 2026-03-21 06:58:57,335 - lava_callback - ERROR - Error processing callback for node 69bb9ea7a1e2bfb1b3347ead: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43000 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59846 - "GET / HTTP/1.0" 200 OK 2026-03-21 06:59:46,657 - lava_callback - INFO - Received callback for node 69bb9ed0a1e2bfb1b3347ebc INFO: 172.21.0.1:59852 - "POST /node/69bb9ed0a1e2bfb1b3347ebc HTTP/1.0" 202 Accepted 2026-03-21 06:59:46,662 - lava_callback - INFO - Starting async job submit for node 69bb9ed0a1e2bfb1b3347ebc 2026-03-21 06:59:46,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:59:46,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb9ed0a1e2bfb1b3347ebc HTTP/1.1" 200 None 2026-03-21 06:59:46,677 - lava_callback - DEBUG - Processing node 69bb9ed0a1e2bfb1b3347ebc: baseline-arm-clabbe 2026-03-21 06:59:46,678 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb9ed0a1e2bfb1b3347ebc 2026-03-21 06:59:46,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:59:46,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb9ed0a1e2bfb1b3347ebc HTTP/1.1" 200 None 2026-03-21 06:59:46,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:59:46,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89da7fbe29c53ae85cce2 HTTP/1.1" 200 None 2026-03-21 06:59:46,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 06:59:46,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb9ed0a1e2bfb1b3347ebc HTTP/1.1" 200 None 2026-03-21 06:59:46,730 - lava_callback - ERROR - Error processing callback for node 69bb9ed0a1e2bfb1b3347ebc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40040 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40052 - "GET / HTTP/1.0" 200 OK 2026-03-21 07:00:17,400 - lava_callback - INFO - Received callback for node 69bb9ef1a1e2bfb1b3347eca INFO: 172.21.0.1:40066 - "POST /node/69bb9ef1a1e2bfb1b3347eca HTTP/1.0" 202 Accepted 2026-03-21 07:00:17,405 - lava_callback - INFO - Starting async job submit for node 69bb9ef1a1e2bfb1b3347eca 2026-03-21 07:00:17,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:00:17,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb9ef1a1e2bfb1b3347eca HTTP/1.1" 200 None 2026-03-21 07:00:17,422 - lava_callback - DEBUG - Processing node 69bb9ef1a1e2bfb1b3347eca: baseline-arm-baylibre 2026-03-21 07:00:17,422 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb9ef1a1e2bfb1b3347eca 2026-03-21 07:00:17,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:00:17,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb9ef1a1e2bfb1b3347eca HTTP/1.1" 200 None 2026-03-21 07:00:17,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:00:17,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89da7fbe29c53ae85cce2 HTTP/1.1" 200 None 2026-03-21 07:00:17,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:00:17,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb9ef1a1e2bfb1b3347eca HTTP/1.1" 200 None 2026-03-21 07:00:17,491 - lava_callback - ERROR - Error processing callback for node 69bb9ef1a1e2bfb1b3347eca: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36626 - "GET / HTTP/1.0" 200 OK 2026-03-21 07:00:57,419 - lava_callback - INFO - Received callback for node 69bb9f16a1e2bfb1b3347edc INFO: 172.21.0.1:36642 - "POST /node/69bb9f16a1e2bfb1b3347edc HTTP/1.0" 202 Accepted 2026-03-21 07:00:57,424 - lava_callback - INFO - Starting async job submit for node 69bb9f16a1e2bfb1b3347edc 2026-03-21 07:00:57,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:00:57,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb9f16a1e2bfb1b3347edc HTTP/1.1" 200 None 2026-03-21 07:00:57,441 - lava_callback - DEBUG - Processing node 69bb9f16a1e2bfb1b3347edc: baseline-arm-baylibre 2026-03-21 07:00:57,441 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb9f16a1e2bfb1b3347edc 2026-03-21 07:00:57,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:00:57,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb9f16a1e2bfb1b3347edc HTTP/1.1" 200 None 2026-03-21 07:00:57,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:00:57,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89da7fbe29c53ae85cce2 HTTP/1.1" 200 None 2026-03-21 07:00:57,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:00:57,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb9f16a1e2bfb1b3347edc HTTP/1.1" 200 None 2026-03-21 07:00:57,509 - lava_callback - ERROR - Error processing callback for node 69bb9f16a1e2bfb1b3347edc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42752 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56180 - "GET / HTTP/1.0" 200 OK 2026-03-21 07:02:06,685 - lava_callback - INFO - Received callback for node 69bb9f60a1e2bfb1b3347ef8 INFO: 172.21.0.1:56188 - "POST /node/69bb9f60a1e2bfb1b3347ef8 HTTP/1.0" 202 Accepted 2026-03-21 07:02:06,690 - lava_callback - INFO - Starting async job submit for node 69bb9f60a1e2bfb1b3347ef8 2026-03-21 07:02:06,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:02:06,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb9f60a1e2bfb1b3347ef8 HTTP/1.1" 200 None 2026-03-21 07:02:06,706 - lava_callback - DEBUG - Processing node 69bb9f60a1e2bfb1b3347ef8: baseline-arm-clabbe 2026-03-21 07:02:06,706 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb9f60a1e2bfb1b3347ef8 2026-03-21 07:02:06,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:02:06,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb9f60a1e2bfb1b3347ef8 HTTP/1.1" 200 None 2026-03-21 07:02:06,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:02:06,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89da7fbe29c53ae85cce2 HTTP/1.1" 200 None 2026-03-21 07:02:06,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:02:06,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb9f60a1e2bfb1b3347ef8 HTTP/1.1" 200 None 2026-03-21 07:02:06,763 - lava_callback - ERROR - Error processing callback for node 69bb9f60a1e2bfb1b3347ef8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33600 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37618 - "GET / HTTP/1.0" 200 OK 2026-03-21 07:02:46,637 - lava_callback - INFO - Received callback for node 69bb9f83a1e2bfb1b3347f07 INFO: 172.21.0.1:37620 - "POST /node/69bb9f83a1e2bfb1b3347f07 HTTP/1.0" 202 Accepted 2026-03-21 07:02:46,642 - lava_callback - INFO - Starting async job submit for node 69bb9f83a1e2bfb1b3347f07 2026-03-21 07:02:46,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:02:46,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb9f83a1e2bfb1b3347f07 HTTP/1.1" 200 None 2026-03-21 07:02:46,660 - lava_callback - DEBUG - Processing node 69bb9f83a1e2bfb1b3347f07: baseline-arm-clabbe 2026-03-21 07:02:46,661 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb9f83a1e2bfb1b3347f07 2026-03-21 07:02:46,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:02:46,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb9f83a1e2bfb1b3347f07 HTTP/1.1" 200 None 2026-03-21 07:02:46,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:02:46,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89da7fbe29c53ae85cce2 HTTP/1.1" 200 None 2026-03-21 07:02:46,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:02:46,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb9f83a1e2bfb1b3347f07 HTTP/1.1" 200 None 2026-03-21 07:02:46,717 - lava_callback - ERROR - Error processing callback for node 69bb9f83a1e2bfb1b3347f07: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58888 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58904 - "GET / HTTP/1.0" 200 OK 2026-03-21 07:03:17,571 - lava_callback - INFO - Received callback for node 69bb9fa6a1e2bfb1b3347f16 INFO: 172.21.0.1:58920 - "POST /node/69bb9fa6a1e2bfb1b3347f16 HTTP/1.0" 202 Accepted 2026-03-21 07:03:17,577 - lava_callback - INFO - Starting async job submit for node 69bb9fa6a1e2bfb1b3347f16 2026-03-21 07:03:17,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:03:17,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb9fa6a1e2bfb1b3347f16 HTTP/1.1" 200 None 2026-03-21 07:03:17,592 - lava_callback - DEBUG - Processing node 69bb9fa6a1e2bfb1b3347f16: baseline-arm-baylibre 2026-03-21 07:03:17,592 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb9fa6a1e2bfb1b3347f16 2026-03-21 07:03:17,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:03:17,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb9fa6a1e2bfb1b3347f16 HTTP/1.1" 200 None 2026-03-21 07:03:17,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:03:17,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89da7fbe29c53ae85cce2 HTTP/1.1" 200 None 2026-03-21 07:03:17,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:03:17,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb9fa6a1e2bfb1b3347f16 HTTP/1.1" 200 None 2026-03-21 07:03:17,652 - lava_callback - ERROR - Error processing callback for node 69bb9fa6a1e2bfb1b3347f16: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57678 - "GET / HTTP/1.0" 200 OK 2026-03-21 07:03:46,701 - lava_callback - INFO - Received callback for node 69bb9fcca1e2bfb1b3347f29 INFO: 172.21.0.1:57686 - "POST /node/69bb9fcca1e2bfb1b3347f29 HTTP/1.0" 202 Accepted 2026-03-21 07:03:46,707 - lava_callback - INFO - Starting async job submit for node 69bb9fcca1e2bfb1b3347f29 2026-03-21 07:03:46,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:03:46,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb9fcca1e2bfb1b3347f29 HTTP/1.1" 200 None 2026-03-21 07:03:46,724 - lava_callback - DEBUG - Processing node 69bb9fcca1e2bfb1b3347f29: baseline-arm-clabbe 2026-03-21 07:03:46,724 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb9fcca1e2bfb1b3347f29 2026-03-21 07:03:46,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:03:46,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb9fcca1e2bfb1b3347f29 HTTP/1.1" 200 None 2026-03-21 07:03:46,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:03:46,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89da7fbe29c53ae85cce2 HTTP/1.1" 200 None 2026-03-21 07:03:46,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:03:46,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb9fcca1e2bfb1b3347f29 HTTP/1.1" 200 None 2026-03-21 07:03:46,776 - lava_callback - ERROR - Error processing callback for node 69bb9fcca1e2bfb1b3347f29: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41240 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55406 - "GET / HTTP/1.0" 200 OK 2026-03-21 07:04:26,691 - lava_callback - INFO - Received callback for node 69bb9ff1a1e2bfb1b3347f40 INFO: 172.21.0.1:55420 - "POST /node/69bb9ff1a1e2bfb1b3347f40 HTTP/1.0" 202 Accepted 2026-03-21 07:04:26,697 - lava_callback - INFO - Starting async job submit for node 69bb9ff1a1e2bfb1b3347f40 2026-03-21 07:04:26,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:04:26,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb9ff1a1e2bfb1b3347f40 HTTP/1.1" 200 None 2026-03-21 07:04:26,718 - lava_callback - DEBUG - Processing node 69bb9ff1a1e2bfb1b3347f40: baseline-arm-clabbe 2026-03-21 07:04:26,719 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bb9ff1a1e2bfb1b3347f40 2026-03-21 07:04:26,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:04:26,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb9ff1a1e2bfb1b3347f40 HTTP/1.1" 200 None 2026-03-21 07:04:26,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:04:26,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89da7fbe29c53ae85cce2 HTTP/1.1" 200 None 2026-03-21 07:04:26,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:04:26,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bb9ff1a1e2bfb1b3347f40 HTTP/1.1" 200 None 2026-03-21 07:04:26,771 - lava_callback - ERROR - Error processing callback for node 69bb9ff1a1e2bfb1b3347f40: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59042 - "GET / HTTP/1.0" 200 OK 2026-03-21 07:05:06,976 - lava_callback - INFO - Received callback for node 69bba015a1e2bfb1b3347f4b INFO: 172.21.0.1:59056 - "POST /node/69bba015a1e2bfb1b3347f4b HTTP/1.0" 202 Accepted 2026-03-21 07:05:06,981 - lava_callback - INFO - Starting async job submit for node 69bba015a1e2bfb1b3347f4b 2026-03-21 07:05:06,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:05:06,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bba015a1e2bfb1b3347f4b HTTP/1.1" 200 None 2026-03-21 07:05:06,998 - lava_callback - DEBUG - Processing node 69bba015a1e2bfb1b3347f4b: baseline-arm-clabbe 2026-03-21 07:05:06,998 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bba015a1e2bfb1b3347f4b 2026-03-21 07:05:06,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:05:07,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bba015a1e2bfb1b3347f4b HTTP/1.1" 200 None 2026-03-21 07:05:07,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:05:07,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89da7fbe29c53ae85cce2 HTTP/1.1" 200 None 2026-03-21 07:05:07,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:05:07,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bba015a1e2bfb1b3347f4b HTTP/1.1" 200 None 2026-03-21 07:05:07,056 - lava_callback - ERROR - Error processing callback for node 69bba015a1e2bfb1b3347f4b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59572 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54262 - "GET / HTTP/1.0" 200 OK 2026-03-21 07:05:46,790 - lava_callback - INFO - Received callback for node 69bba03ba1e2bfb1b3347f5a INFO: 172.21.0.1:54278 - "POST /node/69bba03ba1e2bfb1b3347f5a HTTP/1.0" 202 Accepted 2026-03-21 07:05:46,797 - lava_callback - INFO - Starting async job submit for node 69bba03ba1e2bfb1b3347f5a 2026-03-21 07:05:46,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:05:46,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bba03ba1e2bfb1b3347f5a HTTP/1.1" 200 None 2026-03-21 07:05:46,815 - lava_callback - DEBUG - Processing node 69bba03ba1e2bfb1b3347f5a: baseline-arm-clabbe 2026-03-21 07:05:46,815 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bba03ba1e2bfb1b3347f5a 2026-03-21 07:05:46,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:05:46,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bba03ba1e2bfb1b3347f5a HTTP/1.1" 200 None 2026-03-21 07:05:46,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:05:46,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89da7fbe29c53ae85cce2 HTTP/1.1" 200 None 2026-03-21 07:05:46,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:05:46,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bba03ba1e2bfb1b3347f5a HTTP/1.1" 200 None 2026-03-21 07:05:46,872 - lava_callback - ERROR - Error processing callback for node 69bba03ba1e2bfb1b3347f5a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49008 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55618 - "GET / HTTP/1.0" 200 OK 2026-03-21 07:06:46,901 - lava_callback - INFO - Received callback for node 69bba083a1e2bfb1b3347f85 INFO: 172.21.0.1:55622 - "POST /node/69bba083a1e2bfb1b3347f85 HTTP/1.0" 202 Accepted 2026-03-21 07:06:46,906 - lava_callback - INFO - Starting async job submit for node 69bba083a1e2bfb1b3347f85 2026-03-21 07:06:46,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:06:46,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bba083a1e2bfb1b3347f85 HTTP/1.1" 200 None 2026-03-21 07:06:46,927 - lava_callback - DEBUG - Processing node 69bba083a1e2bfb1b3347f85: baseline-arm-clabbe 2026-03-21 07:06:46,927 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bba083a1e2bfb1b3347f85 2026-03-21 07:06:46,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:06:46,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bba083a1e2bfb1b3347f85 HTTP/1.1" 200 None 2026-03-21 07:06:46,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:06:46,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89da7fbe29c53ae85cce2 HTTP/1.1" 200 None 2026-03-21 07:06:46,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:06:46,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bba083a1e2bfb1b3347f85 HTTP/1.1" 200 None 2026-03-21 07:06:46,992 - lava_callback - ERROR - Error processing callback for node 69bba083a1e2bfb1b3347f85: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50942 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50516 - "GET / HTTP/1.0" 200 OK 2026-03-21 07:07:26,900 - lava_callback - INFO - Received callback for node 69bba0a7a1e2bfb1b3347fa6 INFO: 172.21.0.1:50526 - "POST /node/69bba0a7a1e2bfb1b3347fa6 HTTP/1.0" 202 Accepted 2026-03-21 07:07:26,905 - lava_callback - INFO - Starting async job submit for node 69bba0a7a1e2bfb1b3347fa6 2026-03-21 07:07:26,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:07:26,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bba0a7a1e2bfb1b3347fa6 HTTP/1.1" 200 None 2026-03-21 07:07:26,921 - lava_callback - DEBUG - Processing node 69bba0a7a1e2bfb1b3347fa6: baseline-arm-clabbe 2026-03-21 07:07:26,922 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bba0a7a1e2bfb1b3347fa6 2026-03-21 07:07:26,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:07:26,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bba0a7a1e2bfb1b3347fa6 HTTP/1.1" 200 None 2026-03-21 07:07:26,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:07:26,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89da7fbe29c53ae85cce2 HTTP/1.1" 200 None 2026-03-21 07:07:26,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:07:26,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bba0a7a1e2bfb1b3347fa6 HTTP/1.1" 200 None 2026-03-21 07:07:26,988 - lava_callback - ERROR - Error processing callback for node 69bba0a7a1e2bfb1b3347fa6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50608 - "GET / HTTP/1.0" 200 OK 2026-03-21 07:08:06,916 - lava_callback - INFO - Received callback for node 69bba0cca1e2bfb1b3347fbb INFO: 172.21.0.1:50612 - "POST /node/69bba0cca1e2bfb1b3347fbb HTTP/1.0" 202 Accepted 2026-03-21 07:08:06,922 - lava_callback - INFO - Starting async job submit for node 69bba0cca1e2bfb1b3347fbb 2026-03-21 07:08:06,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:08:06,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bba0cca1e2bfb1b3347fbb HTTP/1.1" 200 None 2026-03-21 07:08:06,938 - lava_callback - DEBUG - Processing node 69bba0cca1e2bfb1b3347fbb: baseline-arm-clabbe 2026-03-21 07:08:06,938 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bba0cca1e2bfb1b3347fbb 2026-03-21 07:08:06,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:08:06,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bba0cca1e2bfb1b3347fbb HTTP/1.1" 200 None 2026-03-21 07:08:06,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:08:06,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89da7fbe29c53ae85cce2 HTTP/1.1" 200 None 2026-03-21 07:08:06,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:08:06,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bba0cca1e2bfb1b3347fbb HTTP/1.1" 200 None 2026-03-21 07:08:06,995 - lava_callback - ERROR - Error processing callback for node 69bba0cca1e2bfb1b3347fbb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52650 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46090 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34456 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41578 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43412 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34914 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52472 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43646 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35942 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51438 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46724 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46728 - "GET / HTTP/1.0" 200 OK 2026-03-21 07:18:18,646 - lava_callback - INFO - Received callback for node 69bba328a1e2bfb1b3348047 INFO: 172.21.0.1:46736 - "POST /node/69bba328a1e2bfb1b3348047 HTTP/1.0" 202 Accepted 2026-03-21 07:18:18,651 - lava_callback - INFO - Starting async job submit for node 69bba328a1e2bfb1b3348047 2026-03-21 07:18:18,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:18:18,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bba328a1e2bfb1b3348047 HTTP/1.1" 200 None 2026-03-21 07:18:18,667 - lava_callback - DEBUG - Processing node 69bba328a1e2bfb1b3348047: baseline-arm64-baylibre 2026-03-21 07:18:18,668 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bba328a1e2bfb1b3348047 2026-03-21 07:18:18,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:18:18,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bba328a1e2bfb1b3348047 HTTP/1.1" 200 None 2026-03-21 07:18:18,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:18:18,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-21 07:18:18,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:18:18,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bba328a1e2bfb1b3348047 HTTP/1.1" 200 None 2026-03-21 07:18:18,720 - lava_callback - ERROR - Error processing callback for node 69bba328a1e2bfb1b3348047: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38774 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51298 - "GET / HTTP/1.0" 200 OK 2026-03-21 07:19:38,447 - lava_callback - INFO - Received callback for node 69bba37ca1e2bfb1b334805c INFO: 172.21.0.1:51304 - "POST /node/69bba37ca1e2bfb1b334805c HTTP/1.0" 202 Accepted 2026-03-21 07:19:38,454 - lava_callback - INFO - Starting async job submit for node 69bba37ca1e2bfb1b334805c 2026-03-21 07:19:38,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:19:38,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bba37ca1e2bfb1b334805c HTTP/1.1" 200 None 2026-03-21 07:19:38,472 - lava_callback - DEBUG - Processing node 69bba37ca1e2bfb1b334805c: baseline-arm64-baylibre 2026-03-21 07:19:38,473 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bba37ca1e2bfb1b334805c 2026-03-21 07:19:38,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:19:38,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bba37ca1e2bfb1b334805c HTTP/1.1" 200 None 2026-03-21 07:19:38,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:19:38,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-21 07:19:38,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:19:38,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bba37ca1e2bfb1b334805c HTTP/1.1" 200 None 2026-03-21 07:19:38,538 - lava_callback - ERROR - Error processing callback for node 69bba37ca1e2bfb1b334805c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41738 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42596 - "GET / HTTP/1.0" 200 OK 2026-03-21 07:20:58,518 - lava_callback - INFO - Received callback for node 69bba3d0a1e2bfb1b334806c INFO: 172.21.0.1:42610 - "POST /node/69bba3d0a1e2bfb1b334806c HTTP/1.0" 202 Accepted 2026-03-21 07:20:58,524 - lava_callback - INFO - Starting async job submit for node 69bba3d0a1e2bfb1b334806c 2026-03-21 07:20:58,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:20:58,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bba3d0a1e2bfb1b334806c HTTP/1.1" 200 None 2026-03-21 07:20:58,571 - lava_callback - DEBUG - Processing node 69bba3d0a1e2bfb1b334806c: baseline-arm64-baylibre 2026-03-21 07:20:58,571 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bba3d0a1e2bfb1b334806c 2026-03-21 07:20:58,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:20:58,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bba3d0a1e2bfb1b334806c HTTP/1.1" 200 None 2026-03-21 07:20:58,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:20:58,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-21 07:20:58,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:20:58,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bba3d0a1e2bfb1b334806c HTTP/1.1" 200 None 2026-03-21 07:20:58,651 - lava_callback - ERROR - Error processing callback for node 69bba3d0a1e2bfb1b334806c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38416 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43360 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43368 - "GET / HTTP/1.0" 200 OK 2026-03-21 07:22:18,598 - lava_callback - INFO - Received callback for node 69bba420a1e2bfb1b3348077 INFO: 172.21.0.1:43370 - "POST /node/69bba420a1e2bfb1b3348077 HTTP/1.0" 202 Accepted 2026-03-21 07:22:18,604 - lava_callback - INFO - Starting async job submit for node 69bba420a1e2bfb1b3348077 2026-03-21 07:22:18,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:22:18,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bba420a1e2bfb1b3348077 HTTP/1.1" 200 None 2026-03-21 07:22:18,630 - lava_callback - DEBUG - Processing node 69bba420a1e2bfb1b3348077: baseline-arm64-baylibre 2026-03-21 07:22:18,630 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bba420a1e2bfb1b3348077 2026-03-21 07:22:18,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:22:18,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bba420a1e2bfb1b3348077 HTTP/1.1" 200 None 2026-03-21 07:22:18,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:22:18,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-21 07:22:18,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:22:18,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bba420a1e2bfb1b3348077 HTTP/1.1" 200 None 2026-03-21 07:22:18,690 - lava_callback - ERROR - Error processing callback for node 69bba420a1e2bfb1b3348077: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54786 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53104 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35980 - "GET / HTTP/1.0" 200 OK 2026-03-21 07:25:07,971 - lava_callback - INFO - Received callback for node 69bba4c9a1e2bfb1b3348097 INFO: 172.21.0.1:35984 - "POST /node/69bba4c9a1e2bfb1b3348097 HTTP/1.0" 202 Accepted 2026-03-21 07:25:07,976 - lava_callback - INFO - Starting async job submit for node 69bba4c9a1e2bfb1b3348097 2026-03-21 07:25:07,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:25:07,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bba4c9a1e2bfb1b3348097 HTTP/1.1" 200 None 2026-03-21 07:25:07,993 - lava_callback - DEBUG - Processing node 69bba4c9a1e2bfb1b3348097: baseline-arm64-clabbe 2026-03-21 07:25:07,993 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bba4c9a1e2bfb1b3348097 2026-03-21 07:25:07,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:25:08,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bba4c9a1e2bfb1b3348097 HTTP/1.1" 200 None 2026-03-21 07:25:08,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:25:08,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-21 07:25:08,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:25:08,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bba4c9a1e2bfb1b3348097 HTTP/1.1" 200 None 2026-03-21 07:25:08,052 - lava_callback - ERROR - Error processing callback for node 69bba4c9a1e2bfb1b3348097: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56390 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45442 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33882 - "GET / HTTP/1.0" 200 OK 2026-03-21 07:26:27,742 - lava_callback - INFO - Received callback for node 69bba51aa1e2bfb1b33480a6 INFO: 172.21.0.1:33892 - "POST /node/69bba51aa1e2bfb1b33480a6 HTTP/1.0" 202 Accepted 2026-03-21 07:26:27,748 - lava_callback - INFO - Starting async job submit for node 69bba51aa1e2bfb1b33480a6 2026-03-21 07:26:27,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:26:27,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bba51aa1e2bfb1b33480a6 HTTP/1.1" 200 None 2026-03-21 07:26:27,765 - lava_callback - DEBUG - Processing node 69bba51aa1e2bfb1b33480a6: baseline-arm64-clabbe 2026-03-21 07:26:27,765 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bba51aa1e2bfb1b33480a6 2026-03-21 07:26:27,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:26:27,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bba51aa1e2bfb1b33480a6 HTTP/1.1" 200 None 2026-03-21 07:26:27,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:26:27,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-21 07:26:27,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:26:27,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bba51aa1e2bfb1b33480a6 HTTP/1.1" 200 None 2026-03-21 07:26:27,841 - lava_callback - ERROR - Error processing callback for node 69bba51aa1e2bfb1b33480a6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45190 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40760 - "GET / HTTP/1.0" 200 OK 2026-03-21 07:27:47,710 - lava_callback - INFO - Received callback for node 69bba56ba1e2bfb1b33480ab INFO: 172.21.0.1:40776 - "POST /node/69bba56ba1e2bfb1b33480ab HTTP/1.0" 202 Accepted 2026-03-21 07:27:47,715 - lava_callback - INFO - Starting async job submit for node 69bba56ba1e2bfb1b33480ab 2026-03-21 07:27:47,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:27:47,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bba56ba1e2bfb1b33480ab HTTP/1.1" 200 None 2026-03-21 07:27:47,732 - lava_callback - DEBUG - Processing node 69bba56ba1e2bfb1b33480ab: baseline-arm64-clabbe 2026-03-21 07:27:47,732 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bba56ba1e2bfb1b33480ab 2026-03-21 07:27:47,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:27:47,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bba56ba1e2bfb1b33480ab HTTP/1.1" 200 None 2026-03-21 07:27:47,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:27:47,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-21 07:27:47,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:27:47,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bba56ba1e2bfb1b33480ab HTTP/1.1" 200 None 2026-03-21 07:27:47,793 - lava_callback - ERROR - Error processing callback for node 69bba56ba1e2bfb1b33480ab: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36248 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49838 - "GET / HTTP/1.0" 200 OK 2026-03-21 07:29:07,790 - lava_callback - INFO - Received callback for node 69bba5b9a1e2bfb1b33480b8 INFO: 172.21.0.1:49840 - "POST /node/69bba5b9a1e2bfb1b33480b8 HTTP/1.0" 202 Accepted 2026-03-21 07:29:07,796 - lava_callback - INFO - Starting async job submit for node 69bba5b9a1e2bfb1b33480b8 2026-03-21 07:29:07,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:29:07,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bba5b9a1e2bfb1b33480b8 HTTP/1.1" 200 None 2026-03-21 07:29:07,815 - lava_callback - DEBUG - Processing node 69bba5b9a1e2bfb1b33480b8: baseline-arm64-clabbe 2026-03-21 07:29:07,816 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bba5b9a1e2bfb1b33480b8 2026-03-21 07:29:07,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:29:07,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bba5b9a1e2bfb1b33480b8 HTTP/1.1" 200 None 2026-03-21 07:29:07,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:29:07,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-21 07:29:07,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:29:07,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bba5b9a1e2bfb1b33480b8 HTTP/1.1" 200 None 2026-03-21 07:29:07,888 - lava_callback - ERROR - Error processing callback for node 69bba5b9a1e2bfb1b33480b8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56150 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44852 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39404 - "GET / HTTP/1.0" 200 OK 2026-03-21 07:30:27,809 - lava_callback - INFO - Received callback for node 69bba610a1e2bfb1b33480c8 INFO: 172.21.0.1:39410 - "POST /node/69bba610a1e2bfb1b33480c8 HTTP/1.0" 202 Accepted 2026-03-21 07:30:27,815 - lava_callback - INFO - Starting async job submit for node 69bba610a1e2bfb1b33480c8 2026-03-21 07:30:27,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:30:27,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bba610a1e2bfb1b33480c8 HTTP/1.1" 200 None 2026-03-21 07:30:27,833 - lava_callback - DEBUG - Processing node 69bba610a1e2bfb1b33480c8: baseline-arm64-clabbe 2026-03-21 07:30:27,833 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bba610a1e2bfb1b33480c8 2026-03-21 07:30:27,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:30:27,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bba610a1e2bfb1b33480c8 HTTP/1.1" 200 None 2026-03-21 07:30:27,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:30:27,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-21 07:30:27,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:30:27,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bba610a1e2bfb1b33480c8 HTTP/1.1" 200 None 2026-03-21 07:30:27,897 - lava_callback - ERROR - Error processing callback for node 69bba610a1e2bfb1b33480c8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45244 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39244 - "GET / HTTP/1.0" 200 OK 2026-03-21 07:32:07,895 - lava_callback - INFO - Received callback for node 69bba664a1e2bfb1b33480d6 INFO: 172.21.0.1:39248 - "POST /node/69bba664a1e2bfb1b33480d6 HTTP/1.0" 202 Accepted 2026-03-21 07:32:07,901 - lava_callback - INFO - Starting async job submit for node 69bba664a1e2bfb1b33480d6 2026-03-21 07:32:07,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:32:07,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bba664a1e2bfb1b33480d6 HTTP/1.1" 200 None 2026-03-21 07:32:07,924 - lava_callback - DEBUG - Processing node 69bba664a1e2bfb1b33480d6: baseline-arm64-clabbe 2026-03-21 07:32:07,924 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bba664a1e2bfb1b33480d6 2026-03-21 07:32:07,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:32:07,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bba664a1e2bfb1b33480d6 HTTP/1.1" 200 None 2026-03-21 07:32:07,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:32:07,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-21 07:32:07,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:32:07,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bba664a1e2bfb1b33480d6 HTTP/1.1" 200 None 2026-03-21 07:32:07,983 - lava_callback - ERROR - Error processing callback for node 69bba664a1e2bfb1b33480d6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57986 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36122 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36136 - "GET / HTTP/1.0" 200 OK 2026-03-21 07:33:19,195 - lava_callback - INFO - Received callback for node 69bba6aba1e2bfb1b33480dd INFO: 172.21.0.1:36150 - "POST /node/69bba6aba1e2bfb1b33480dd HTTP/1.0" 202 Accepted 2026-03-21 07:33:19,201 - lava_callback - INFO - Starting async job submit for node 69bba6aba1e2bfb1b33480dd 2026-03-21 07:33:19,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:33:19,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bba6aba1e2bfb1b33480dd HTTP/1.1" 200 None 2026-03-21 07:33:19,216 - lava_callback - DEBUG - Processing node 69bba6aba1e2bfb1b33480dd: baseline-arm64-baylibre 2026-03-21 07:33:19,217 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bba6aba1e2bfb1b33480dd 2026-03-21 07:33:19,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:33:19,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bba6aba1e2bfb1b33480dd HTTP/1.1" 200 None 2026-03-21 07:33:19,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:33:19,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-21 07:33:19,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:33:19,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bba6aba1e2bfb1b33480dd HTTP/1.1" 200 None 2026-03-21 07:33:19,282 - lava_callback - ERROR - Error processing callback for node 69bba6aba1e2bfb1b33480dd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56418 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45248 - "GET / HTTP/1.0" 200 OK 2026-03-21 07:34:39,229 - lava_callback - INFO - Received callback for node 69bba6fca1e2bfb1b33480ec INFO: 172.21.0.1:45254 - "POST /node/69bba6fca1e2bfb1b33480ec HTTP/1.0" 202 Accepted 2026-03-21 07:34:39,235 - lava_callback - INFO - Starting async job submit for node 69bba6fca1e2bfb1b33480ec 2026-03-21 07:34:39,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:34:39,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bba6fca1e2bfb1b33480ec HTTP/1.1" 200 None 2026-03-21 07:34:39,251 - lava_callback - DEBUG - Processing node 69bba6fca1e2bfb1b33480ec: baseline-arm64-baylibre 2026-03-21 07:34:39,251 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bba6fca1e2bfb1b33480ec 2026-03-21 07:34:39,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:34:39,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bba6fca1e2bfb1b33480ec HTTP/1.1" 200 None 2026-03-21 07:34:39,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:34:39,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-21 07:34:39,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:34:39,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bba6fca1e2bfb1b33480ec HTTP/1.1" 200 None 2026-03-21 07:34:39,313 - lava_callback - ERROR - Error processing callback for node 69bba6fca1e2bfb1b33480ec: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50258 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37750 - "GET / HTTP/1.0" 200 OK 2026-03-21 07:35:59,270 - lava_callback - INFO - Received callback for node 69bba74ca1e2bfb1b33480f9 INFO: 172.21.0.1:37758 - "POST /node/69bba74ca1e2bfb1b33480f9 HTTP/1.0" 202 Accepted 2026-03-21 07:35:59,275 - lava_callback - INFO - Starting async job submit for node 69bba74ca1e2bfb1b33480f9 2026-03-21 07:35:59,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:35:59,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bba74ca1e2bfb1b33480f9 HTTP/1.1" 200 None 2026-03-21 07:35:59,296 - lava_callback - DEBUG - Processing node 69bba74ca1e2bfb1b33480f9: baseline-arm64-baylibre 2026-03-21 07:35:59,296 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bba74ca1e2bfb1b33480f9 2026-03-21 07:35:59,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:35:59,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bba74ca1e2bfb1b33480f9 HTTP/1.1" 200 None 2026-03-21 07:35:59,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:35:59,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-21 07:35:59,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:35:59,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bba74ca1e2bfb1b33480f9 HTTP/1.1" 200 None 2026-03-21 07:35:59,371 - lava_callback - ERROR - Error processing callback for node 69bba74ca1e2bfb1b33480f9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59112 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39758 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39768 - "GET / HTTP/1.0" 200 OK 2026-03-21 07:37:19,359 - lava_callback - INFO - Received callback for node 69bba79da1e2bfb1b334810a INFO: 172.21.0.1:39780 - "POST /node/69bba79da1e2bfb1b334810a HTTP/1.0" 202 Accepted 2026-03-21 07:37:19,365 - lava_callback - INFO - Starting async job submit for node 69bba79da1e2bfb1b334810a 2026-03-21 07:37:19,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:37:19,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bba79da1e2bfb1b334810a HTTP/1.1" 200 None 2026-03-21 07:37:19,389 - lava_callback - DEBUG - Processing node 69bba79da1e2bfb1b334810a: baseline-arm64-baylibre 2026-03-21 07:37:19,389 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bba79da1e2bfb1b334810a 2026-03-21 07:37:19,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:37:19,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bba79da1e2bfb1b334810a HTTP/1.1" 200 None 2026-03-21 07:37:19,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:37:19,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-21 07:37:19,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:37:19,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bba79da1e2bfb1b334810a HTTP/1.1" 200 None 2026-03-21 07:37:19,451 - lava_callback - ERROR - Error processing callback for node 69bba79da1e2bfb1b334810a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49090 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35734 - "GET / HTTP/1.0" 200 OK 2026-03-21 07:38:39,495 - lava_callback - INFO - Received callback for node 69bba7eca1e2bfb1b334811c INFO: 172.21.0.1:35740 - "POST /node/69bba7eca1e2bfb1b334811c HTTP/1.0" 202 Accepted 2026-03-21 07:38:39,500 - lava_callback - INFO - Starting async job submit for node 69bba7eca1e2bfb1b334811c 2026-03-21 07:38:39,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:38:39,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bba7eca1e2bfb1b334811c HTTP/1.1" 200 None 2026-03-21 07:38:39,516 - lava_callback - DEBUG - Processing node 69bba7eca1e2bfb1b334811c: baseline-arm64-baylibre 2026-03-21 07:38:39,517 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bba7eca1e2bfb1b334811c 2026-03-21 07:38:39,517 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:38:39,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bba7eca1e2bfb1b334811c HTTP/1.1" 200 None 2026-03-21 07:38:39,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:38:39,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-21 07:38:39,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:38:39,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bba7eca1e2bfb1b334811c HTTP/1.1" 200 None 2026-03-21 07:38:39,573 - lava_callback - ERROR - Error processing callback for node 69bba7eca1e2bfb1b334811c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52258 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35086 - "GET / HTTP/1.0" 200 OK 2026-03-21 07:39:59,543 - lava_callback - INFO - Received callback for node 69bba83da1e2bfb1b3348125 INFO: 172.21.0.1:35088 - "POST /node/69bba83da1e2bfb1b3348125 HTTP/1.0" 202 Accepted 2026-03-21 07:39:59,548 - lava_callback - INFO - Starting async job submit for node 69bba83da1e2bfb1b3348125 2026-03-21 07:39:59,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:39:59,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bba83da1e2bfb1b3348125 HTTP/1.1" 200 None 2026-03-21 07:39:59,570 - lava_callback - DEBUG - Processing node 69bba83da1e2bfb1b3348125: baseline-arm64-baylibre 2026-03-21 07:39:59,570 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bba83da1e2bfb1b3348125 2026-03-21 07:39:59,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:39:59,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bba83da1e2bfb1b3348125 HTTP/1.1" 200 None 2026-03-21 07:39:59,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:39:59,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-21 07:39:59,599 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:39:59,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bba83da1e2bfb1b3348125 HTTP/1.1" 200 None 2026-03-21 07:39:59,631 - lava_callback - ERROR - Error processing callback for node 69bba83da1e2bfb1b3348125: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53730 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44758 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35166 - "GET / HTTP/1.0" 200 OK 2026-03-21 07:41:28,277 - lava_callback - INFO - Received callback for node 69bba895a1e2bfb1b3348133 INFO: 172.21.0.1:35180 - "POST /node/69bba895a1e2bfb1b3348133 HTTP/1.0" 202 Accepted 2026-03-21 07:41:28,283 - lava_callback - INFO - Starting async job submit for node 69bba895a1e2bfb1b3348133 2026-03-21 07:41:28,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:41:28,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bba895a1e2bfb1b3348133 HTTP/1.1" 200 None 2026-03-21 07:41:28,300 - lava_callback - DEBUG - Processing node 69bba895a1e2bfb1b3348133: baseline-arm64-clabbe 2026-03-21 07:41:28,300 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bba895a1e2bfb1b3348133 2026-03-21 07:41:28,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:41:28,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bba895a1e2bfb1b3348133 HTTP/1.1" 200 None 2026-03-21 07:41:28,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:41:28,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-21 07:41:28,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:41:28,351 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bba895a1e2bfb1b3348133 HTTP/1.1" 200 None 2026-03-21 07:41:28,357 - lava_callback - ERROR - Error processing callback for node 69bba895a1e2bfb1b3348133: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-21 07:41:34,420 - lava_callback - INFO - Received callback for node 69bd7add0e93e2314723e735 INFO: 172.21.0.1:45430 - "POST /node/69bd7add0e93e2314723e735 HTTP/1.0" 202 Accepted 2026-03-21 07:41:34,425 - lava_callback - INFO - Starting async job submit for node 69bd7add0e93e2314723e735 2026-03-21 07:41:34,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:41:34,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd7add0e93e2314723e735 HTTP/1.1" 200 None 2026-03-21 07:41:34,463 - lava_callback - DEBUG - Processing node 69bd7add0e93e2314723e735: baseline-arm64-broonie 2026-03-21 07:41:34,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 07:41:34,917 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 07:41:34,919 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69bd7add0e93e2314723e735/log.txt.gz 2026-03-21 07:41:35,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 07:41:35,315 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 07:41:35,317 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69bd7add0e93e2314723e735/lava-logs.yaml 2026-03-21 07:41:35,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 07:41:35,456 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 07:41:35,458 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69bd7add0e93e2314723e735/lava_callback.json.gz 2026-03-21 07:41:35,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:41:35,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd7add0e93e2314723e735 HTTP/1.1" 200 None 2026-03-21 07:41:35,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:41:35,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-21 07:41:35,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:41:35,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd7add0e93e2314723e735 HTTP/1.1" 200 None 2026-03-21 07:41:35,539 - lava_callback - INFO - Completed processing callback for node 69bd7add0e93e2314723e735 INFO: 172.21.0.1:45436 - "GET / HTTP/1.0" 200 OK 2026-03-21 07:41:37,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:41:37,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:46656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54886 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44644 - "GET / HTTP/1.0" 200 OK 2026-03-21 07:42:48,315 - lava_callback - INFO - Received callback for node 69bba8e5a1e2bfb1b3348147 INFO: 172.21.0.1:44646 - "POST /node/69bba8e5a1e2bfb1b3348147 HTTP/1.0" 202 Accepted 2026-03-21 07:42:48,322 - lava_callback - INFO - Starting async job submit for node 69bba8e5a1e2bfb1b3348147 2026-03-21 07:42:48,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:42:48,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bba8e5a1e2bfb1b3348147 HTTP/1.1" 200 None 2026-03-21 07:42:48,349 - lava_callback - DEBUG - Processing node 69bba8e5a1e2bfb1b3348147: baseline-arm64-clabbe 2026-03-21 07:42:48,350 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bba8e5a1e2bfb1b3348147 2026-03-21 07:42:48,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:42:48,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bba8e5a1e2bfb1b3348147 HTTP/1.1" 200 None 2026-03-21 07:42:48,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:42:48,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-21 07:42:48,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:42:48,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bba8e5a1e2bfb1b3348147 HTTP/1.1" 200 None 2026-03-21 07:42:48,403 - lava_callback - ERROR - Error processing callback for node 69bba8e5a1e2bfb1b3348147: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56104 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36114 - "GET / HTTP/1.0" 200 OK 2026-03-21 07:43:34,524 - lava_callback - INFO - Received callback for node 69bdd6160e93e23147242cef INFO: 172.21.0.1:56276 - "POST /node/69bdd6160e93e23147242cef HTTP/1.0" 202 Accepted 2026-03-21 07:43:34,530 - lava_callback - INFO - Starting async job submit for node 69bdd6160e93e23147242cef 2026-03-21 07:43:34,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:43:34,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd6160e93e23147242cef HTTP/1.1" 200 None 2026-03-21 07:43:34,568 - lava_callback - DEBUG - Processing node 69bdd6160e93e23147242cef: baseline-arm64-broonie 2026-03-21 07:43:34,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 07:43:34,991 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 07:43:34,993 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69bdd6160e93e23147242cef/log.txt.gz 2026-03-21 07:43:35,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 07:43:35,416 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 07:43:35,418 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69bdd6160e93e23147242cef/lava-logs.yaml 2026-03-21 07:43:35,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 07:43:35,583 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 07:43:35,585 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69bdd6160e93e23147242cef/lava_callback.json.gz 2026-03-21 07:43:35,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:43:35,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd6160e93e23147242cef HTTP/1.1" 200 None 2026-03-21 07:43:35,625 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:43:35,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-21 07:43:35,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:43:35,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdd6160e93e23147242cef HTTP/1.1" 200 None 2026-03-21 07:43:35,670 - lava_callback - INFO - Completed processing callback for node 69bdd6160e93e23147242cef INFO: 172.21.0.1:56278 - "GET / HTTP/1.0" 200 OK 2026-03-21 07:43:37,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:43:37,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:46318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55116 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51224 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55062 - "GET / HTTP/1.0" 200 OK 2026-03-21 07:45:35,170 - lava_callback - INFO - Received callback for node 69bdf4bed54b5122738c6a55 INFO: 172.21.0.1:58296 - "POST /node/69bdf4bed54b5122738c6a55 HTTP/1.0" 202 Accepted 2026-03-21 07:45:35,176 - lava_callback - INFO - Starting async job submit for node 69bdf4bed54b5122738c6a55 2026-03-21 07:45:35,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:45:35,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf4bed54b5122738c6a55 HTTP/1.1" 200 None 2026-03-21 07:45:35,306 - lava_callback - DEBUG - Processing node 69bdf4bed54b5122738c6a55: baseline-arm64-broonie 2026-03-21 07:45:35,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 07:45:35,732 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 07:45:35,733 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69bdf4bed54b5122738c6a55/log.txt.gz 2026-03-21 07:45:35,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 07:45:36,131 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 07:45:36,133 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69bdf4bed54b5122738c6a55/lava-logs.yaml 2026-03-21 07:45:36,140 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 07:45:36,276 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 07:45:36,278 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69bdf4bed54b5122738c6a55/lava_callback.json.gz 2026-03-21 07:45:36,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:45:36,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdf4bed54b5122738c6a55 HTTP/1.1" 200 None 2026-03-21 07:45:36,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:45:36,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-21 07:45:36,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:45:36,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdf4bed54b5122738c6a55 HTTP/1.1" 200 None 2026-03-21 07:45:36,382 - lava_callback - INFO - Completed processing callback for node 69bdf4bed54b5122738c6a55 INFO: 172.21.0.1:58312 - "GET / HTTP/1.0" 200 OK 2026-03-21 07:45:37,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:45:37,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:55458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46876 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54134 - "GET / HTTP/1.0" 200 OK 2026-03-21 07:46:28,500 - lava_callback - INFO - Received callback for node 69bba9cba1e2bfb1b3348174 INFO: 172.21.0.1:54148 - "POST /node/69bba9cba1e2bfb1b3348174 HTTP/1.0" 202 Accepted 2026-03-21 07:46:28,505 - lava_callback - INFO - Starting async job submit for node 69bba9cba1e2bfb1b3348174 2026-03-21 07:46:28,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:46:28,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bba9cba1e2bfb1b3348174 HTTP/1.1" 200 None 2026-03-21 07:46:28,533 - lava_callback - DEBUG - Processing node 69bba9cba1e2bfb1b3348174: baseline-arm64-clabbe 2026-03-21 07:46:28,533 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bba9cba1e2bfb1b3348174 2026-03-21 07:46:28,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:46:28,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bba9cba1e2bfb1b3348174 HTTP/1.1" 200 None 2026-03-21 07:46:28,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:46:28,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-21 07:46:28,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:46:28,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bba9cba1e2bfb1b3348174 HTTP/1.1" 200 None 2026-03-21 07:46:28,593 - lava_callback - ERROR - Error processing callback for node 69bba9cba1e2bfb1b3348174: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48768 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34060 - "GET / HTTP/1.0" 200 OK 2026-03-21 07:47:34,667 - lava_callback - INFO - Received callback for node 69be2aa8d54b5122738c72b7 INFO: 172.21.0.1:59618 - "POST /node/69be2aa8d54b5122738c72b7 HTTP/1.0" 202 Accepted 2026-03-21 07:47:34,673 - lava_callback - INFO - Starting async job submit for node 69be2aa8d54b5122738c72b7 2026-03-21 07:47:34,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:47:34,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be2aa8d54b5122738c72b7 HTTP/1.1" 200 None 2026-03-21 07:47:34,710 - lava_callback - DEBUG - Processing node 69be2aa8d54b5122738c72b7: baseline-arm64-broonie 2026-03-21 07:47:34,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 07:47:35,135 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 07:47:35,136 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69be2aa8d54b5122738c72b7/log.txt.gz 2026-03-21 07:47:35,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 07:47:35,518 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 07:47:35,520 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69be2aa8d54b5122738c72b7/lava-logs.yaml 2026-03-21 07:47:35,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 07:47:35,666 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 07:47:35,668 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69be2aa8d54b5122738c72b7/lava_callback.json.gz 2026-03-21 07:47:35,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:47:35,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be2aa8d54b5122738c72b7 HTTP/1.1" 200 None 2026-03-21 07:47:35,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:47:35,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-21 07:47:35,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:47:35,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be2aa8d54b5122738c72b7 HTTP/1.1" 200 None 2026-03-21 07:47:35,757 - lava_callback - INFO - Completed processing callback for node 69be2aa8d54b5122738c72b7 INFO: 172.21.0.1:59630 - "GET / HTTP/1.0" 200 OK 2026-03-21 07:47:37,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:47:37,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-21 07:47:39,868 - lava_callback - INFO - Received callback for node 69bbaa13a1e2bfb1b334817a INFO: 172.21.0.1:39552 - "POST /node/69bbaa13a1e2bfb1b334817a HTTP/1.0" 202 Accepted 2026-03-21 07:47:39,874 - lava_callback - INFO - Starting async job submit for node 69bbaa13a1e2bfb1b334817a 2026-03-21 07:47:39,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:47:39,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbaa13a1e2bfb1b334817a HTTP/1.1" 200 None 2026-03-21 07:47:39,892 - lava_callback - DEBUG - Processing node 69bbaa13a1e2bfb1b334817a: baseline-arm64-baylibre 2026-03-21 07:47:39,892 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbaa13a1e2bfb1b334817a 2026-03-21 07:47:39,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:47:39,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbaa13a1e2bfb1b334817a HTTP/1.1" 200 None 2026-03-21 07:47:39,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:47:39,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-21 07:47:39,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:47:39,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbaa13a1e2bfb1b334817a HTTP/1.1" 200 None 2026-03-21 07:47:39,955 - lava_callback - ERROR - Error processing callback for node 69bbaa13a1e2bfb1b334817a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43112 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55444 - "GET / HTTP/1.0" 200 OK 2026-03-21 07:49:08,645 - lava_callback - INFO - Received callback for node 69bbaa6aa1e2bfb1b334818b INFO: 172.21.0.1:55450 - "POST /node/69bbaa6aa1e2bfb1b334818b HTTP/1.0" 202 Accepted 2026-03-21 07:49:08,650 - lava_callback - INFO - Starting async job submit for node 69bbaa6aa1e2bfb1b334818b 2026-03-21 07:49:08,651 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:49:08,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbaa6aa1e2bfb1b334818b HTTP/1.1" 200 None 2026-03-21 07:49:08,666 - lava_callback - DEBUG - Processing node 69bbaa6aa1e2bfb1b334818b: baseline-arm64-clabbe 2026-03-21 07:49:08,666 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbaa6aa1e2bfb1b334818b 2026-03-21 07:49:08,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:49:08,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbaa6aa1e2bfb1b334818b HTTP/1.1" 200 None 2026-03-21 07:49:08,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:49:08,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-21 07:49:08,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:49:08,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbaa6aa1e2bfb1b334818b HTTP/1.1" 200 None 2026-03-21 07:49:08,718 - lava_callback - ERROR - Error processing callback for node 69bbaa6aa1e2bfb1b334818b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57178 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60590 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54864 - "GET / HTTP/1.0" 200 OK 2026-03-21 07:50:28,627 - lava_callback - INFO - Received callback for node 69bbaabaa1e2bfb1b334819f INFO: 172.21.0.1:54866 - "POST /node/69bbaabaa1e2bfb1b334819f HTTP/1.0" 202 Accepted 2026-03-21 07:50:28,632 - lava_callback - INFO - Starting async job submit for node 69bbaabaa1e2bfb1b334819f 2026-03-21 07:50:28,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:50:28,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbaabaa1e2bfb1b334819f HTTP/1.1" 200 None 2026-03-21 07:50:28,649 - lava_callback - DEBUG - Processing node 69bbaabaa1e2bfb1b334819f: baseline-arm64-clabbe 2026-03-21 07:50:28,650 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbaabaa1e2bfb1b334819f 2026-03-21 07:50:28,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:50:28,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbaabaa1e2bfb1b334819f HTTP/1.1" 200 None 2026-03-21 07:50:28,665 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:50:28,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-21 07:50:28,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:50:28,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbaabaa1e2bfb1b334819f HTTP/1.1" 200 None 2026-03-21 07:50:28,718 - lava_callback - ERROR - Error processing callback for node 69bbaabaa1e2bfb1b334819f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58310 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56320 - "GET / HTTP/1.0" 200 OK 2026-03-21 07:51:40,131 - lava_callback - INFO - Received callback for node 69bbab00a1e2bfb1b33481a8 INFO: 172.21.0.1:39618 - "POST /node/69bbab00a1e2bfb1b33481a8 HTTP/1.0" 202 Accepted 2026-03-21 07:51:40,137 - lava_callback - INFO - Starting async job submit for node 69bbab00a1e2bfb1b33481a8 2026-03-21 07:51:40,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:51:40,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbab00a1e2bfb1b33481a8 HTTP/1.1" 200 None 2026-03-21 07:51:40,153 - lava_callback - DEBUG - Processing node 69bbab00a1e2bfb1b33481a8: baseline-arm64-baylibre 2026-03-21 07:51:40,154 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbab00a1e2bfb1b33481a8 2026-03-21 07:51:40,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:51:40,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbab00a1e2bfb1b33481a8 HTTP/1.1" 200 None 2026-03-21 07:51:40,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:51:40,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-21 07:51:40,186 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:51:40,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbab00a1e2bfb1b33481a8 HTTP/1.1" 200 None 2026-03-21 07:51:40,218 - lava_callback - ERROR - Error processing callback for node 69bbab00a1e2bfb1b33481a8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46986 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59178 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51036 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34118 - "GET / HTTP/1.0" 200 OK 2026-03-21 07:54:28,931 - lava_callback - INFO - Received callback for node 69bbaba2a1e2bfb1b33481c2 INFO: 172.21.0.1:34134 - "POST /node/69bbaba2a1e2bfb1b33481c2 HTTP/1.0" 202 Accepted 2026-03-21 07:54:28,936 - lava_callback - INFO - Starting async job submit for node 69bbaba2a1e2bfb1b33481c2 2026-03-21 07:54:28,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:54:28,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbaba2a1e2bfb1b33481c2 HTTP/1.1" 200 None 2026-03-21 07:54:28,952 - lava_callback - DEBUG - Processing node 69bbaba2a1e2bfb1b33481c2: baseline-arm64-clabbe 2026-03-21 07:54:28,953 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbaba2a1e2bfb1b33481c2 2026-03-21 07:54:28,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:54:28,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbaba2a1e2bfb1b33481c2 HTTP/1.1" 200 None 2026-03-21 07:54:28,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:54:28,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-21 07:54:28,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:54:29,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbaba2a1e2bfb1b33481c2 HTTP/1.1" 200 None 2026-03-21 07:54:29,006 - lava_callback - ERROR - Error processing callback for node 69bbaba2a1e2bfb1b33481c2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50840 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56160 - "GET / HTTP/1.0" 200 OK 2026-03-21 07:55:40,582 - lava_callback - INFO - Received callback for node 69bbabe9a1e2bfb1b33481d2 INFO: 172.21.0.1:34166 - "POST /node/69bbabe9a1e2bfb1b33481d2 HTTP/1.0" 202 Accepted 2026-03-21 07:55:40,587 - lava_callback - INFO - Starting async job submit for node 69bbabe9a1e2bfb1b33481d2 2026-03-21 07:55:40,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:55:40,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbabe9a1e2bfb1b33481d2 HTTP/1.1" 200 None 2026-03-21 07:55:40,604 - lava_callback - DEBUG - Processing node 69bbabe9a1e2bfb1b33481d2: baseline-arm64-baylibre 2026-03-21 07:55:40,604 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbabe9a1e2bfb1b33481d2 2026-03-21 07:55:40,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:55:40,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbabe9a1e2bfb1b33481d2 HTTP/1.1" 200 None 2026-03-21 07:55:40,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:55:40,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-21 07:55:40,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:55:40,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbabe9a1e2bfb1b33481d2 HTTP/1.1" 200 None 2026-03-21 07:55:40,662 - lava_callback - ERROR - Error processing callback for node 69bbabe9a1e2bfb1b33481d2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48776 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43398 - "GET / HTTP/1.0" 200 OK 2026-03-21 07:57:08,981 - lava_callback - INFO - Received callback for node 69bbac3fa1e2bfb1b33481e5 INFO: 172.21.0.1:43404 - "POST /node/69bbac3fa1e2bfb1b33481e5 HTTP/1.0" 202 Accepted 2026-03-21 07:57:08,986 - lava_callback - INFO - Starting async job submit for node 69bbac3fa1e2bfb1b33481e5 2026-03-21 07:57:08,987 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:57:09,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbac3fa1e2bfb1b33481e5 HTTP/1.1" 200 None 2026-03-21 07:57:09,003 - lava_callback - DEBUG - Processing node 69bbac3fa1e2bfb1b33481e5: baseline-arm64-clabbe 2026-03-21 07:57:09,003 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbac3fa1e2bfb1b33481e5 2026-03-21 07:57:09,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:57:09,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbac3fa1e2bfb1b33481e5 HTTP/1.1" 200 None 2026-03-21 07:57:09,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:57:09,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-21 07:57:09,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:57:09,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbac3fa1e2bfb1b33481e5 HTTP/1.1" 200 None 2026-03-21 07:57:09,058 - lava_callback - ERROR - Error processing callback for node 69bbac3fa1e2bfb1b33481e5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55360 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37350 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34200 - "GET / HTTP/1.0" 200 OK 2026-03-21 07:58:28,972 - lava_callback - INFO - Received callback for node 69bbac8ea1e2bfb1b33481ec INFO: 172.21.0.1:34204 - "POST /node/69bbac8ea1e2bfb1b33481ec HTTP/1.0" 202 Accepted 2026-03-21 07:58:28,978 - lava_callback - INFO - Starting async job submit for node 69bbac8ea1e2bfb1b33481ec 2026-03-21 07:58:28,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:58:29,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbac8ea1e2bfb1b33481ec HTTP/1.1" 200 None 2026-03-21 07:58:29,004 - lava_callback - DEBUG - Processing node 69bbac8ea1e2bfb1b33481ec: baseline-arm64-clabbe 2026-03-21 07:58:29,004 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbac8ea1e2bfb1b33481ec 2026-03-21 07:58:29,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:58:29,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbac8ea1e2bfb1b33481ec HTTP/1.1" 200 None 2026-03-21 07:58:29,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:58:29,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-21 07:58:29,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:58:29,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbac8ea1e2bfb1b33481ec HTTP/1.1" 200 None 2026-03-21 07:58:29,064 - lava_callback - ERROR - Error processing callback for node 69bbac8ea1e2bfb1b33481ec: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56926 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55212 - "GET / HTTP/1.0" 200 OK 2026-03-21 07:59:29,016 - lava_callback - INFO - Received callback for node 69bbacdea1e2bfb1b33481f9 INFO: 172.21.0.1:55228 - "POST /node/69bbacdea1e2bfb1b33481f9 HTTP/1.0" 202 Accepted 2026-03-21 07:59:29,022 - lava_callback - INFO - Starting async job submit for node 69bbacdea1e2bfb1b33481f9 2026-03-21 07:59:29,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:59:29,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbacdea1e2bfb1b33481f9 HTTP/1.1" 200 None 2026-03-21 07:59:29,038 - lava_callback - DEBUG - Processing node 69bbacdea1e2bfb1b33481f9: baseline-arm64-clabbe 2026-03-21 07:59:29,039 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbacdea1e2bfb1b33481f9 2026-03-21 07:59:29,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:59:29,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbacdea1e2bfb1b33481f9 HTTP/1.1" 200 None 2026-03-21 07:59:29,054 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:59:29,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-21 07:59:29,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 07:59:29,093 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbacdea1e2bfb1b33481f9 HTTP/1.1" 200 None 2026-03-21 07:59:29,098 - lava_callback - ERROR - Error processing callback for node 69bbacdea1e2bfb1b33481f9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53338 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46088 - "GET / HTTP/1.0" 200 OK 2026-03-21 08:00:49,108 - lava_callback - INFO - Received callback for node 69bbad2ea1e2bfb1b3348208 INFO: 172.21.0.1:46090 - "POST /node/69bbad2ea1e2bfb1b3348208 HTTP/1.0" 202 Accepted 2026-03-21 08:00:49,113 - lava_callback - INFO - Starting async job submit for node 69bbad2ea1e2bfb1b3348208 2026-03-21 08:00:49,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:00:49,129 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbad2ea1e2bfb1b3348208 HTTP/1.1" 200 None 2026-03-21 08:00:49,130 - lava_callback - DEBUG - Processing node 69bbad2ea1e2bfb1b3348208: baseline-arm64-clabbe 2026-03-21 08:00:49,130 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbad2ea1e2bfb1b3348208 2026-03-21 08:00:49,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:00:49,145 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbad2ea1e2bfb1b3348208 HTTP/1.1" 200 None 2026-03-21 08:00:49,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:00:49,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-21 08:00:49,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:00:49,185 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbad2ea1e2bfb1b3348208 HTTP/1.1" 200 None 2026-03-21 08:00:49,189 - lava_callback - ERROR - Error processing callback for node 69bbad2ea1e2bfb1b3348208: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42666 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41436 - "GET / HTTP/1.0" 200 OK 2026-03-21 08:02:09,127 - lava_callback - INFO - Received callback for node 69bbad7da1e2bfb1b3348211 INFO: 172.21.0.1:41438 - "POST /node/69bbad7da1e2bfb1b3348211 HTTP/1.0" 202 Accepted 2026-03-21 08:02:09,132 - lava_callback - INFO - Starting async job submit for node 69bbad7da1e2bfb1b3348211 2026-03-21 08:02:09,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:02:09,149 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbad7da1e2bfb1b3348211 HTTP/1.1" 200 None 2026-03-21 08:02:09,150 - lava_callback - DEBUG - Processing node 69bbad7da1e2bfb1b3348211: baseline-arm64-clabbe 2026-03-21 08:02:09,151 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbad7da1e2bfb1b3348211 2026-03-21 08:02:09,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:02:09,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbad7da1e2bfb1b3348211 HTTP/1.1" 200 None 2026-03-21 08:02:09,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:02:09,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-21 08:02:09,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:02:09,203 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbad7da1e2bfb1b3348211 HTTP/1.1" 200 None 2026-03-21 08:02:09,204 - lava_callback - ERROR - Error processing callback for node 69bbad7da1e2bfb1b3348211: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57774 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42948 - "GET / HTTP/1.0" 200 OK 2026-03-21 08:02:55,261 - lava_callback - INFO - Received callback for node 69be502ad54b5122738c7963 INFO: 172.21.0.1:38922 - "POST /node/69be502ad54b5122738c7963 HTTP/1.0" 202 Accepted 2026-03-21 08:02:55,267 - lava_callback - INFO - Starting async job submit for node 69be502ad54b5122738c7963 2026-03-21 08:02:55,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:02:55,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be502ad54b5122738c7963 HTTP/1.1" 200 None 2026-03-21 08:02:55,313 - lava_callback - DEBUG - Processing node 69be502ad54b5122738c7963: baseline-arm64-broonie 2026-03-21 08:02:55,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:02:55,741 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:02:55,742 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69be502ad54b5122738c7963/log.txt.gz 2026-03-21 08:02:55,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:02:56,148 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:02:56,150 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69be502ad54b5122738c7963/lava-logs.yaml 2026-03-21 08:02:56,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-21 08:02:56,342 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-21 08:02:56,344 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69be502ad54b5122738c7963/lava_callback.json.gz 2026-03-21 08:02:56,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:02:56,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be502ad54b5122738c7963 HTTP/1.1" 200 None 2026-03-21 08:02:56,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:02:56,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-21 08:02:56,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:02:56,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69be502ad54b5122738c7963 HTTP/1.1" 200 None 2026-03-21 08:02:56,423 - lava_callback - INFO - Completed processing callback for node 69be502ad54b5122738c7963 INFO: 172.21.0.1:38928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49464 - "GET / HTTP/1.0" 200 OK 2026-03-21 08:03:07,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:03:07,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:41832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41844 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51190 - "GET / HTTP/1.0" 200 OK 2026-03-21 08:03:29,265 - lava_callback - INFO - Received callback for node 69bbadcea1e2bfb1b334821e INFO: 172.21.0.1:51202 - "POST /node/69bbadcea1e2bfb1b334821e HTTP/1.0" 202 Accepted 2026-03-21 08:03:29,271 - lava_callback - INFO - Starting async job submit for node 69bbadcea1e2bfb1b334821e 2026-03-21 08:03:29,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:03:29,286 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbadcea1e2bfb1b334821e HTTP/1.1" 200 None 2026-03-21 08:03:29,286 - lava_callback - DEBUG - Processing node 69bbadcea1e2bfb1b334821e: baseline-arm64-clabbe 2026-03-21 08:03:29,287 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bbadcea1e2bfb1b334821e 2026-03-21 08:03:29,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:03:29,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbadcea1e2bfb1b334821e HTTP/1.1" 200 None 2026-03-21 08:03:29,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:03:29,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-21 08:03:29,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-21 08:03:29,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bbadcea1e2bfb1b334821e HTTP/1.1" 200 None 2026-03-21 08:03:29,351 - lava_callback - ERROR - Error processing callback for node 69bbadcea1e2bfb1b334821e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41978 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57898 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49956 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35246 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34960 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33130 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47916 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59742 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44378 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41854 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40802 - "GET / HTTP/1.0" 200 OK INFO: Shutting down INFO: Waiting for application shutdown. INFO: Application shutdown complete. INFO: Finished server process [1]